Calling a REST API with Python and requests: Practical Examples
How to use the requests library to read and send data to a REST API in Python: GET and POST requests, JSON, authentication, errors and automatic retries.
Many of the services we use every day, from weather forecasts to business management software, make their data available through an API. Knowing how to use one from Python lets you automate a huge number of tasks: downloading data, synchronising two services, publishing content, generating reports.
In this guide we use requests, the most popular library for the job, with examples you can run straight away against free public APIs.
What is a REST API, in brief
An API (Application Programming Interface) is a way for one program to ask another program for data or actions. REST APIs work over the web, using the same protocol as websites, HTTP. There are only a few concepts to know:
- Endpoint: the address the request is sent to, for example
https://api.example.com/customers. - Method: the type of operation. The main ones are
GET(read),POST(create),PUTorPATCH(modify) andDELETE(delete). - JSON: the text format usually used to exchange data. In Python it becomes a dictionary or a list.
- Status code: a number indicating the outcome.
200and201mean success;4xxindicates an error in the request (for example401unauthorised,404not found,429too many requests);5xxindicates a server problem.
Installing requests
As always, it is best to work inside a virtual environment. With the environment active:
python -m pip install requests
The first GET request: the weather forecast
We use Open-Meteo, a free weather service that requires no registration. Let's ask for the current weather in Milan:
import requests
URL = "https://api.open-meteo.com/v1/forecast"
params = {
"latitude": 45.46,
"longitude": 9.19,
"current": "temperature_2m,wind_speed_10m",
"timezone": "auto",
}
response = requests.get(URL, params=params, timeout=10)
response.raise_for_status()
data = response.json()
current = data["current"]
print(f"Temperature: {current['temperature_2m']} °C")
print(f"Wind: {current['wind_speed_10m']} km/h")
Let's look at the important lines:
params: requests adds the parameters to the address in the correct format, also handling spaces and special characters. There is no need to build the URL by hand.timeout=10: if the server does not respond within 10 seconds, the request is aborted. Without a timeout, a script can hang forever. Always set one.raise_for_status(): if the status code indicates an error, it raises an exception instead of carrying on with wrong data.json(): turns the response into a Python dictionary.
A practical tip: when working with a new API, print the full response with print(data) to see its structure before extracting the fields you are interested in.
Sending data with a POST request
To create something you use the POST method. To try it without risk, we use JSONPlaceholder, a test service that simulates a real API but saves nothing:
import requests
new_post = {
"title": "Test",
"body": "Test content",
"userId": 1,
}
response = requests.post(
"https://jsonplaceholder.typicode.com/posts",
json=new_post,
timeout=10,
)
response.raise_for_status()
print(response.status_code) # 201: created
print(response.json()) # the data received, with the assigned id
The json= parameter converts the dictionary to JSON and automatically sets the Content-Type: application/json header. It is different from data=, which sends the data like a web form: if an API expects JSON, always use json=.
Authentication with a token
Private APIs require you to identify yourself. The most common method is a token (or API key) sent in the request headers:
import os
import requests
token = os.environ["API_TOKEN"]
headers = {
"Authorization": f"Bearer {token}",
"Accept": "application/json",
}
response = requests.get(
"https://api.example.com/v1/customers",
headers=headers,
timeout=10,
)
Note that the token is not written in the code: it is read from an environment variable. This is an important security rule: if the code ends up on GitHub or is shared, the token stays private. To set the variable before running the script:
# Windows, Command Prompt
set API_TOKEN=your-token
# Windows, PowerShell
$env:API_TOKEN = "your-token"
# Linux and macOS
export API_TOKEN="your-token"
The exact header format (Bearer, Token, a key in a dedicated header) depends on the API: you will always find it in its documentation.
Handling errors
In a real script the network may not respond, the server may be under maintenance, the token may expire. With a try block you can handle each case with a clear message:
import requests
try:
response = requests.get(URL, params=params, timeout=10)
response.raise_for_status()
except requests.exceptions.Timeout:
print("The server did not respond in time.")
except requests.exceptions.HTTPError as error:
print(f"HTTP error {error.response.status_code}: {error.response.text[:200]}")
except requests.exceptions.ConnectionError:
print("Connection failed: check the network or the address.")
else:
data = response.json()
print(data)
The else block runs only if there were no errors. Printing part of the response text (error.response.text) is useful, because many APIs explain the reason for the error there.
Sessions and automatic retries
If the script makes many requests to the same service, it is worth using a session: it reuses the connection (so it is faster) and lets you set common headers and behaviour just once. With a session you can also automatically retry requests that failed because of temporary problems:
import requests
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry
retries = Retry(
total=3, # up to 3 new attempts
backoff_factor=1, # increasing waits between attempts
status_forcelist=[429, 500, 502, 503, 504],
allowed_methods=["GET"], # only retry reads
)
session = requests.Session()
session.mount("https://", HTTPAdapter(max_retries=retries))
session.headers.update({"User-Agent": "my-script/1.0"})
response = session.get(URL, params=params, timeout=10)
response.raise_for_status()
Why retry only GET requests? Because reading twice does no harm, whereas repeating a POST could create the same item twice, for example two identical orders.
Best practices
- Always set a timeout, on every request.
- Check the status code with
raise_for_status()before using the data. - Never write tokens and passwords in the code: use environment variables.
- Respect the API's limits: many APIs limit the number of requests. If you get a
429error, slow down; some indicate in theRetry-Afterheader how many seconds to wait. - Handle pagination: APIs that return long lists split them into pages. The documentation tells you which parameter to use (for example
pageorcursor) to request the next pages. - Read the API documentation: addresses, parameters and authentication format vary from one service to another.
What about httpx?
You may also come across httpx, a more recent library with an interface very similar to requests, which also supports asynchronous requests (that is, many requests in parallel). For scripts and automation, requests remains the simplest choice; if one day you need asynchronous features, moving to httpx will feel natural.
Official sources
Share
What's Your Reaction?
Like
0
Dislike
0
Love
0
Funny
0
Angry
0
Sad
0
Wow
0