Calling a REST API with Python and requests: Practical Examples

How to use the requests library to read and send data to a REST API in Python: GET and POST requests, JSON, authentication, errors and automatic retries.

Calling a REST API with Python and requests: Practical Examples

Many of the services we use every day, from weather forecasts to business management software, make their data available through an API. Knowing how to use one from Python lets you automate a huge number of tasks: downloading data, synchronising two services, publishing content, generating reports.

In this guide we use requests, the most popular library for the job, with examples you can run straight away against free public APIs.

What is a REST API, in brief

An API (Application Programming Interface) is a way for one program to ask another program for data or actions. REST APIs work over the web, using the same protocol as websites, HTTP. There are only a few concepts to know:

  • Endpoint: the address the request is sent to, for example https://api.example.com/customers.
  • Method: the type of operation. The main ones are GET (read), POST (create), PUT or PATCH (modify) and DELETE (delete).
  • JSON: the text format usually used to exchange data. In Python it becomes a dictionary or a list.
  • Status code: a number indicating the outcome. 200 and 201 mean success; 4xx indicates an error in the request (for example 401 unauthorised, 404 not found, 429 too many requests); 5xx indicates a server problem.

Installing requests

As always, it is best to work inside a virtual environment. With the environment active:

python -m pip install requests

The first GET request: the weather forecast

We use Open-Meteo, a free weather service that requires no registration. Let's ask for the current weather in Milan:

import requests

URL = "https://api.open-meteo.com/v1/forecast"
params = {
    "latitude": 45.46,
    "longitude": 9.19,
    "current": "temperature_2m,wind_speed_10m",
    "timezone": "auto",
}

response = requests.get(URL, params=params, timeout=10)
response.raise_for_status()

data = response.json()
current = data["current"]
print(f"Temperature: {current['temperature_2m']} °C")
print(f"Wind: {current['wind_speed_10m']} km/h")

Let's look at the important lines:

  • params: requests adds the parameters to the address in the correct format, also handling spaces and special characters. There is no need to build the URL by hand.
  • timeout=10: if the server does not respond within 10 seconds, the request is aborted. Without a timeout, a script can hang forever. Always set one.
  • raise_for_status(): if the status code indicates an error, it raises an exception instead of carrying on with wrong data.
  • json(): turns the response into a Python dictionary.

A practical tip: when working with a new API, print the full response with print(data) to see its structure before extracting the fields you are interested in.

Sending data with a POST request

To create something you use the POST method. To try it without risk, we use JSONPlaceholder, a test service that simulates a real API but saves nothing:

import requests

new_post = {
    "title": "Test",
    "body": "Test content",
    "userId": 1,
}

response = requests.post(
    "https://jsonplaceholder.typicode.com/posts",
    json=new_post,
    timeout=10,
)
response.raise_for_status()

print(response.status_code)   # 201: created
print(response.json())        # the data received, with the assigned id

The json= parameter converts the dictionary to JSON and automatically sets the Content-Type: application/json header. It is different from data=, which sends the data like a web form: if an API expects JSON, always use json=.

Authentication with a token

Private APIs require you to identify yourself. The most common method is a token (or API key) sent in the request headers:

import os
import requests

token = os.environ["API_TOKEN"]
headers = {
    "Authorization": f"Bearer {token}",
    "Accept": "application/json",
}

response = requests.get(
    "https://api.example.com/v1/customers",
    headers=headers,
    timeout=10,
)

Note that the token is not written in the code: it is read from an environment variable. This is an important security rule: if the code ends up on GitHub or is shared, the token stays private. To set the variable before running the script:

# Windows, Command Prompt
set API_TOKEN=your-token

# Windows, PowerShell
$env:API_TOKEN = "your-token"

# Linux and macOS
export API_TOKEN="your-token"

The exact header format (Bearer, Token, a key in a dedicated header) depends on the API: you will always find it in its documentation.

Handling errors

In a real script the network may not respond, the server may be under maintenance, the token may expire. With a try block you can handle each case with a clear message:

import requests

try:
    response = requests.get(URL, params=params, timeout=10)
    response.raise_for_status()
except requests.exceptions.Timeout:
    print("The server did not respond in time.")
except requests.exceptions.HTTPError as error:
    print(f"HTTP error {error.response.status_code}: {error.response.text[:200]}")
except requests.exceptions.ConnectionError:
    print("Connection failed: check the network or the address.")
else:
    data = response.json()
    print(data)

The else block runs only if there were no errors. Printing part of the response text (error.response.text) is useful, because many APIs explain the reason for the error there.

Sessions and automatic retries

If the script makes many requests to the same service, it is worth using a session: it reuses the connection (so it is faster) and lets you set common headers and behaviour just once. With a session you can also automatically retry requests that failed because of temporary problems:

import requests
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry

retries = Retry(
    total=3,                     # up to 3 new attempts
    backoff_factor=1,            # increasing waits between attempts
    status_forcelist=[429, 500, 502, 503, 504],
    allowed_methods=["GET"],     # only retry reads
)

session = requests.Session()
session.mount("https://", HTTPAdapter(max_retries=retries))
session.headers.update({"User-Agent": "my-script/1.0"})

response = session.get(URL, params=params, timeout=10)
response.raise_for_status()

Why retry only GET requests? Because reading twice does no harm, whereas repeating a POST could create the same item twice, for example two identical orders.

Best practices

  • Always set a timeout, on every request.
  • Check the status code with raise_for_status() before using the data.
  • Never write tokens and passwords in the code: use environment variables.
  • Respect the API's limits: many APIs limit the number of requests. If you get a 429 error, slow down; some indicate in the Retry-After header how many seconds to wait.
  • Handle pagination: APIs that return long lists split them into pages. The documentation tells you which parameter to use (for example page or cursor) to request the next pages.
  • Read the API documentation: addresses, parameters and authentication format vary from one service to another.

What about httpx?

You may also come across httpx, a more recent library with an interface very similar to requests, which also supports asynchronous requests (that is, many requests in parallel). For scripts and automation, requests remains the simplest choice; if one day you need asynchronous features, moving to httpx will feel natural.

Official sources

Share

What's Your Reaction?

Like Like 0
Dislike Dislike 0
Love Love 0
Funny Funny 0
Angry Angry 0
Sad Sad 0
Wow Wow 0