python - Call API for each element in list

Question

I have a list with over 1000 IDs and I want to call an API with different endpoints for every element of the list. Example:

customerlist = [803818, 803808, 803803,803738,803730]

I tried the following:

import json
import requests
import pandas as pd



API_BASEURL = "https://exampleurl.com/"
API_TOKEN = "abc"
HEADERS = {'content-type' : 'application/json',
           'Authorization': API_TOKEN }


def get_data(endpoint):
    for i in customerlist:
        api_endpoint = endpoint
        params = {'customerid' : i}
        response = requests.get(f"{API_BASEURL}/{api_endpoint}",
                             params = params,
                             headers = HEADERS)
        if response.status_code == 200:
            res = json.loads(response.text)
        else:
            raise Exception(f'API error with status code {response.status_code}')
        res= pd.DataFrame([res])
        return res

get_data(endpointexample)

This works, but it only returns the values for the first element of the list (803818). I want the function to return the values for every ID from customerlist for the endpoint I defined in the function argument.

I found this - possibly related - question, but I couldn't figure my problem out.

There is probably an easy solution for this which I am not seeing, as I am just starting with Python. Thanks.

Answer 1

The moment a function hits a return statement, it immediately finishes. Since your return statement is in the loop, the other iterations never actually get called.

To fix, you can create a list outside the loop, append to it every loop iteration, and then return the DataFrame created with that list:

def get_data(endpoint):
    responses = []
    for i in customerlist:
        api_endpoint = endpoint
        params = {'customerid' : i}
        response = requests.get(f"{API_BASEURL}/{api_endpoint}",
                             params = params,
                             headers = HEADERS)
        if response.status_code == 200:
            res = json.loads(response.text)
        else:
            raise Exception(f'API error with status code {response.status_code}')
        responses.append(res)
    return pd.DataFrame(responses)

A much cleaner solution would be to use list comprehension:

def get_data(endpoint, i):
    api_endpoint = endpoint
    params = {'customerid' : i}
    response = requests.get(f"{API_BASEURL}/{api_endpoint}",
                         params = params,
                         headers = HEADERS)
    if response.status_code == 200:
        res = json.loads(response.text)
    else:
        raise Exception(f'API error with status code {response.status_code}')

    return res

responses = pd.DataFrame([get_data(endpoint, i) for i in customerlist])