Skip to content
Tutorials Center
My learning

Topics

Lesson 40 of 50 ยท Testing and production

Performance and caching

Make a FastAPI app faster: measure first, avoid N+1 queries, cap page sizes, cache with a TTL, send ETag and Cache-Control headers and tune pools and JSON.

"Is my API fast?" is a question you can only answer with numbers. Most slow endpoints are not slow because Python is slow. They are slow because they ask the database fifty questions instead of one, return ten thousand rows nobody asked for, or recompute the same answer for every visitor. FastAPI itself adds very little overhead per request; what you do inside the endpoint dominates.

This lesson gives you a short, practical order of attack: measure, fix the queries, limit how much data a request can pull, cache what rarely changes, let clients and proxies cache with HTTP headers, and only then tune the small things such as the JSON encoder and connection pools.

The order of attack

Measure Find the slow part first.
Fix queries Remove N+1 and add indexes.
Limit Cap page size and payload.
Cache Reuse answers that rarely change.
1 / 2

Measure before you change anything

Guessing wastes time. The simplest tool is time.perf_counter(), a high-resolution clock meant for measuring durations. Wrap the code you suspect, or write a middleware that times every request (see Middleware and CORS). For whole-service numbers use a load-testing tool such as wrk, hey or Locust against a real server, and a profiler such as py-spy to see where Python spends time.

Timing numbers differ on every machine and every run, so the examples in this lesson count operations (queries, cache misses) wherever they can. Counting is deterministic, and it is usually the real cause of slowness anyway.

Python
import time

start = time.perf_counter()
result = do_expensive_thing()
elapsed_ms = (time.perf_counter() - start) * 1000
print(f"took {elapsed_ms:.1f} ms")

The N+1 query problem

Imagine an endpoint that lists authors with their books. The straightforward code fetches the authors (1 query), then loops and fetches each author's books (N queries). With 5 authors that is 6 queries; with 500 it is 501, each paying a round trip to the database. That is the N+1 problem, and it is the most common reason a list endpoint is slow.

The cure is to fetch related data in bulk: a JOIN, or one extra query with WHERE author_id IN (...). ORMs offer the same idea under names like eager loading (selectinload in SQLAlchemy, covered in CRUD, filtering and pagination).

Python Counting queries: N+1 versus one query

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Both endpoints return identical JSON, but one issues six queries and the other two. The gap grows with every author you add, and the fast version stays at two no matter how many authors there are. Make it a habit to log or count queries in tests for list endpoints: a test that asserts "at most 3 queries" catches an N+1 regression the day it is introduced.

Limit what a request can ask for

An endpoint that returns "all the rows" works on your laptop with 20 rows and falls over in production with 2 million. Always paginate, and always put an upper bound on the page size so a client cannot ask for ?limit=1000000. Validation does it for you:

Python
from typing import Annotated
from fastapi import Query

@app.get("/items")
async def list_items(
    limit: Annotated[int, Query(ge=1, le=100)] = 20,
    offset: Annotated[int, Query(ge=0)] = 0,
): ...

A request with limit=5000 is rejected with a 422 before your code runs. Do the same for upload sizes and list lengths inside request bodies (see API keys and security practices). Beyond limits, two cheap wins help big responses: return only the fields clients need (a smaller response_model) and enable GZipMiddleware, covered in the middleware lesson.

Cache what rarely changes

If thousands of users request the same product list and it changes once an hour, computing it thousands of times is waste. A cache stores an answer and reuses it. The simplest useful cache is a dictionary with a time-to-live (TTL): each entry remembers when it expires.

The example below injects the clock so that the output is exactly reproducible. It also shows the HTTP side of caching: the response carries Cache-Control (how long anyone may reuse it) and an ETag (a fingerprint of the content). A client that already has the data sends the fingerprint back in If-None-Match; if it still matches, the server answers 304 Not Modified with no body.

Python A TTL cache plus ETag and Cache-Control

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The server computed the catalog only twice for four requests: once at the start and once after the 60-second TTL ran out. The revalidation request cost almost nothing, because a 304 carries no body.

Two cautions apply to every cache:

  • Invalidation. When the data changes, decide how the cache learns about it: wait for the TTL, delete the key on write, or both. A stale price is a bug.
  • Scope. An in-process dictionary belongs to one worker process. With four workers you have four caches, each warming separately. For a shared cache use Redis, and never cache per-user data under a key that does not include the user.

Libraries such as fastapi-cache2 wrap the same ideas in decorators, and functools.lru_cache is fine for pure functions with a few distinct arguments.

Smaller wins: JSON, models and pools

  • Response validation costs a little. With response_model, FastAPI validates and re-serializes what you return. That is valuable (it hides secret fields, see Response models and nested data) and cheap for small payloads, but for huge lists it is measurable. If a profile shows it matters, return already-built Response/JSONResponse objects for that one hot endpoint.
  • Faster JSON. The orjson library encodes JSON several times faster than the standard library. Newer FastAPI versions already serialize through Pydantic's Rust core when you declare a return type, so a custom response class is rarely needed; measure first. Read-only sketch:
Python
# pip install orjson
import orjson
from fastapi import Response

@app.get("/big")
async def big() -> Response:
    return Response(content=orjson.dumps(build_big_payload()), media_type="application/json")
  • Connection pools. Opening a database connection is slow, so engines keep a pool of open ones. SQLAlchemy's default pool holds 5 connections with 10 more allowed temporarily (pool_size, max_overflow). The total across all worker processes must stay below the database's own connection limit: 4 workers with the defaults can open up to 60. Create the engine once in the lifespan (Lifespan events and app state).
  • Do not block the event loop. A slow synchronous call inside async def stalls every request on that worker. See async def vs def endpoints.
  • Add indexes. A column you filter or sort by usually needs one; check the query plan with EXPLAIN.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What is the N+1 query problem?

    The loop issues one query per row, so the number of round trips grows with the data. A JOIN or an IN query fetches everything at once.
  2. 2. Why should a page-size parameter have an upper bound such as le=100?

    Without a ceiling, ?limit=1000000 would load a million rows. The validation rejects it with a 422 before your code runs.
  3. 3. A client sends If-None-Match with the ETag it already has, and the content has not changed. What should the server reply?

    A 304 tells the client to reuse its copy, saving bandwidth and serialization work.
  4. 4. Why is an in-process dictionary cache not enough when you run several worker processes?

    Memory is not shared between processes. Use a shared store such as Redis when all workers must see the same cache.

Summary

  • Measure first (perf_counter, load tests, profilers); in tests, count queries rather than milliseconds.
  • Remove N+1 patterns with joins or IN queries, and add indexes for filtered and sorted columns.
  • Paginate everything and bound the page size with Query(le=...).
  • A TTL cache avoids recomputation; plan how entries are invalidated and remember that each worker has its own memory.
  • Cache-Control and ETag let browsers and proxies reuse responses and answer 304 without a body.
  • Size connection pools across all workers, never block the event loop, and reach for orjson only when a profile says so.

Next, put the app on a server in Deployment.