I appear to have exhausted the internet searching for what feels like a common occurrence, and I need some help, please.

I'm making an API call using the requests library, which returns one JSON response per call - I'm going to loop through and make multiple calls.

I want to combine all of the responses from the many API calls into one python data structure and then export the results to CSV.

One API response looks like this:

{
"status": "1",
"msg": "Success",
"data": {
"id": "12345",
"PriceDetail": [
{
"item": "Apple",
"amount": "10",
"weight": "225",
"price": "92",
"bestbeforeendeate": "30/09/2023"
}
]
}
}

My final output should be a CSV file with the following headers and data in the subsequent rows:

iditemamountweightpricebestbeforeendeate
12345apple102259230/09/2023
...........................

I've looked at combining the responses in a dictionary, named tuple, dataframe and tried the various options to export to from said structures like dictwriter, csvwriter, normalize etc. Still, I'm struggling to make any of it work.

The closest I got was (I saved the results to a JSON file to stop hitting the API):

with open('item.json') as json_file: 
data_set = json.load(json_file)
for data in data_set:
if data['msg'] == 'Success':
id = data['data']['id']
return_data[id] = data['data']['PriceDetail']

df = pd.json_normalize(data['data']['PriceDetail'])
print(df)

I couldn't get the id added to the dataframe

Any suggestions would be appreciated.

Thanks,