python - JSON object must be str, not Response

Question

I'm making a request to a URL that returns a JSON file,

response = requests.get(url)
response = json.loads(response)

And then, I am attempting to go through some of the data,

for documents in response["docs"]:
    # do something

Right now, the error I'm getting is

TypeError: the JSON object must be str, not 'Response'

In order to avoid this, I tried,

response = requests.get(url).json()

But then, I can't traverse the response because I get the error:

KeyError: 'docs'

I'm new to Python and not fully aware of the best way to get JSON data and parse it. Suggestions?

Here is a sample of the data being received,

{'status': 'OK', 'response': {'meta': {'time': 9, 'hits': 11, 'offset': 0}, 'docs': [{'type_of_material': 'News', 'pub_date': '2017-01-01T09:12:04+0000', 'document_type': 'article', '_id': '5868c7e995d0e03926078885', 'lead_paragraph': 'The country’s leader spoke proudly of the progress of its nuclear weapons and ballistic missile programs.', .....

Answer 1

You are trying to feed the response object to json.loads(). You don't get a string there, you'd have to access the .contents or .text attribute instead:

response = requests.get(url)
# Python 3
response = json.loads(response.text)
# Python 2
response = json.loads(response.content)

However, that'd be doing more work than you need to; requests supports handling JSON directly, and there is no need to import the json module:

response = requests.get(url).json()

See the JSON Response Content section of the requests Quickstart documentation.

Once loaded, you can get the doc key in the nested dictionary keyed on response:

for documents in response['response']['docs']: