Skip to content
Tutorials Center
My learning

Topics

Lesson 32 of 40 ยท Async Libraries and Patterns

Async HTTP Clients and Calling APIs Concurrently

Learn to call many web APIs concurrently with asyncio, using aiohttp or httpx patterns: shared sessions, timeouts, gather, error handling and a runnable fake-API stand-in.

The classic reason to learn asyncio is making many HTTP requests at once. Fetching 100 URLs one after another takes 100 round trips. Fetching them concurrently takes roughly the time of the slowest few, because while one request waits for the network, the others make progress.

This lesson shows the two popular async HTTP libraries, aiohttp and httpx, and the patterns that matter more than the library: one shared client, timeouts, concurrent calls with gather, limits on concurrency, and sensible error handling.

Honesty box. aiohttp and httpx are third-party packages that open real network connections, which the browser-based lesson editor cannot do. The library snippets below are read-only and you should run them on your own computer after pip install aiohttp httpx. The runnable examples use a small FakeAPI class with the same shape (an async get that takes time and may fail), so you can practise all the surrounding asyncio patterns here.

aiohttp in one screen

Python
import asyncio
import aiohttp

async def fetch_json(session, url):
    async with session.get(url) as response:
        response.raise_for_status()          # 4xx/5xx become exceptions
        return await response.json()

async def main():
    timeout = aiohttp.ClientTimeout(total=10, connect=3)
    async with aiohttp.ClientSession(timeout=timeout) as session:
        urls = [f"https://api.example.org/items/{i}" for i in range(20)]
        results = await asyncio.gather(*(fetch_json(session, u) for u in urls))
    print(len(results), "items")

asyncio.run(main())

Three things to notice:

  1. ClientSession is created once and reused. It owns a connection pool, so repeated requests to the same host reuse open TCP/TLS connections. Creating a session per request throws that away and is both slow and wasteful.
  2. async with session.get(...) makes sure the response is released back to the pool.
  3. The reading of the body is a second awaited step (await response.json()), because the headers can arrive before the body does.

httpx in one screen

httpx offers a very similar API for both sync and async code, which many people like because it resembles requests:

Python
import asyncio
import httpx

async def main():
    async with httpx.AsyncClient(timeout=10.0, base_url="https://api.example.org") as client:
        responses = await asyncio.gather(*(client.get(f"/items/{i}") for i in range(20)))
        data = [r.json() for r in responses if r.status_code == 200]
    print(len(data), "items")

asyncio.run(main())

Both libraries support streaming bodies, cookies, redirects, proxies and HTTP/2 (httpx with an extra, aiohttp through other tools). Pick either; the patterns below are identical.

A fake API to practise with

FakeAPI.get(url) waits a number of event-loop turns (its "latency") and returns a response dictionary. Some paths always fail, so we can practise error handling. It deliberately behaves like a real async client: await api.get(...) is the only entry point.

Python Concurrent calls with gather

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Sequential cost is close to the sum of the latencies (30 + 60 + 20), concurrent cost close to the maximum (60); the few extra turns are the loop's own bookkeeping. Also note gather returns results in the order you passed the awaitables, not the order they finished.

Handling failures

By default gather raises the first exception and leaves the other requests running, which is rarely what you want for a batch of independent calls. Use return_exceptions=True to get every outcome back, exceptions included, then sort them out:

Python Partial failure with return_exceptions

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The two failure shapes are different and both matter: transport errors (the request never completed: timeouts, resets, DNS) arrive as exceptions, while HTTP errors (404, 500) are valid responses that you must check, or convert to exceptions with raise_for_status().

Limiting concurrency

Firing 10,000 requests at once will exhaust file descriptors, trigger rate limits and get you blocked. Put a ceiling on how many are in flight with an asyncio.Semaphore. The next lesson, rate limiting and concurrency limits, covers this in depth. Real clients also have their own limits (aiohttp.TCPConnector(limit=100), httpx.Limits(max_connections=100)).

Timeouts, retries and cancellation

Every network call needs a timeout. The libraries have their own (ClientTimeout, httpx.Timeout), which separate connect, read and total time; or wrap the call in asyncio.timeout() as in the timeouts lesson. For flaky endpoints add retries with backoff, and only for idempotent requests such as GET.

When a task awaiting a request is cancelled, the library closes that connection and the cancellation propagates normally, as in cancelling tasks. That is a big advantage of async HTTP: abandoning a slow request is cheap.

Practical checklist

  • One long-lived session or client per program (or per downstream service), closed with async with.
  • Set total and connect timeouts explicitly. Library defaults can be very long or even unlimited.
  • Limit concurrency with a semaphore or the client's connection limit.
  • Handle both exceptions and bad status codes; log the URL with each failure.
  • Respect 429 Too Many Requests and the Retry-After header.
  • Stream large downloads (async for chunk in response.content.iter_chunked(65536)) instead of reading everything into memory, and combine with backpressure when writing to a slow sink.
  • For tests, replace the client with a fake as we did here, or use a mocking library for the transport.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why should you create one ClientSession or AsyncClient and reuse it?

    The session owns the connection pool. Creating one per request wastes TCP and TLS handshakes.
  2. 2. Three requests take 3, 6 and 2 seconds. About how long does asyncio.gather of the three take?

    Concurrent calls overlap, so the total is roughly the longest one, not the sum.
  3. 3. What does gather(..., return_exceptions=True) change?

    Each result is either a value or an exception object, so one failure does not hide the other outcomes.
  4. 4. A server replies with HTTP 404. How does a library like aiohttp report it by default?

    An HTTP error status is still a complete response. Check the status yourself or call raise_for_status() to turn it into an exception.

Summary

  • Concurrent requests with gather cost about as much as the slowest one, not the sum.
  • Share one session or client, set timeouts, and bound concurrency.
  • Collect partial results with return_exceptions=True, and treat transport errors and HTTP error statuses separately.
  • The libraries (aiohttp, httpx) are read-only here; the fake API lets you practise every surrounding pattern.

Next: Rate limiting and concurrency limits