Skip to content
Tutorials Center
My learning

Topics

Lesson 38 of 50 ยท Databases

MongoDB with FastAPI (PyMongo async and Motor)

Connect FastAPI to MongoDB with the async PyMongo client (or Motor): create the client in lifespan, handle ObjectId with Pydantic, and build CRUD endpoints you can run here.

Not every project wants tables. MongoDB stores documents (JSON-like objects) in collections, which suits data whose shape varies or nests deeply: product catalogs, content, event logs. The SQLAlchemy and SQLModel lessons used a relational database; this one shows the document-database version of the same ideas.

Two Python drivers matter. PyMongo is the official driver and now ships a native async client, AsyncMongoClient. Motor was the older async wrapper around PyMongo, and it is deprecated in favour of PyMongo's async API. Their method names are nearly identical (await collection.insert_one(...)), so everything in this lesson applies to both. New code should start with from pymongo import AsyncMongoClient.

A MongoDB server cannot run inside the browser, so the runnable examples use mongomock-motor, an in-memory stand-in with the same async API. The code that is specific to a real server (the connection URL and client creation) is shown read-only and marked as such.

Connecting: one client for the whole app

A MongoDB client manages its own connection pool and is meant to be created once and shared, never per request. That is exactly what the lifespan hook is for: create the client at startup, keep it on app.state, close it on shutdown.

Python
# Read-only: needs a running MongoDB server
from contextlib import asynccontextmanager
from fastapi import FastAPI
from pymongo import AsyncMongoClient

@asynccontextmanager
async def lifespan(app: FastAPI):
    app.state.mongo = AsyncMongoClient("mongodb://localhost:27017")
    app.state.db = app.state.mongo["shop"]
    yield
    await app.state.mongo.close()

app = FastAPI(lifespan=lifespan)

Read the connection string from settings (an environment variable), never from source code. For a hosted cluster it looks like mongodb+srv://user:[email protected]/?retryWrites=true. With Motor, the line would be AsyncIOMotorClient(url) and shutdown client.close() without await; the rest is unchanged.

Then expose the collection through a dependency, so endpoints ask for what they need and tests can replace it:

Python
from fastapi import Depends, Request

async def get_items(request: Request):
    return request.app.state.db["items"]

The ObjectId problem

Every MongoDB document has a unique _id. By default it is an ObjectId: a 12-byte value usually written as 24 hex characters, such as 6af293c09e0004b7fb23903f. It is not a JSON type, and Pydantic does not know it, which is the first stumbling block with FastAPI. Two facts solve it:

  • Output: convert the ObjectId to a string. A Pydantic BeforeValidator(str) does that while the document is validated.
  • Input: an id that arrives in a URL is a string. Turn it into an ObjectId only when you query, and reject malformed ones first (ObjectId("bad") raises InvalidId, which would otherwise become a 500 error).
Python
from typing import Annotated
from bson import ObjectId
from pydantic import BaseModel, BeforeValidator, Field

PyObjectId = Annotated[str, BeforeValidator(str)]

class ItemIn(BaseModel):
    name: str = Field(min_length=1)
    price: float = Field(gt=0)
    tags: list[str] = []

class ItemOut(ItemIn):
    id: PyObjectId = Field(validation_alias="_id")

validation_alias="_id" means "fill id from the document's _id key", while the JSON you send back uses the readable name id. (A separate input and output model is the same pattern as in Response models: the client never supplies the id.)

CRUD endpoints

The five operations map onto the driver like this:

Operation Method Returns
Create insert_one(doc) result with inserted_id
Read one find_one({"_id": oid}) the document or None
Read many find(filter).skip(n).limit(m).to_list() list of documents
Update update_one(filter, {"$set": {...}}) result with matched_count
Delete delete_one(filter) result with deleted_count

The following program is a complete little product API. It runs against the in-memory stand-in; in a real project you would swap the client in the lifespan function and nothing else.

Python A product API on MongoDB (in-memory stand-in)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Several details are worth studying:

  • The id in the JSON is a plain string even though MongoDB stored an ObjectId; the BeforeValidator(str) did the conversion.
  • A malformed id (not-an-id) is answered with 404 by the valid_object_id dependency; the driver is never asked.
  • update_one uses $set with only the fields the client sent (exclude_unset=True), the same idea as in Partial updates. matched_count tells you whether the document existed, even if nothing changed.
  • skip and limit give simple pagination. For large collections, prefer range pagination: filter on the last seen _id ({"_id": {"$gt": last_id}}) and sort("_id"), because large skip values make the server walk past every skipped document.
  • .to_list(length=limit) is the async way to collect a cursor. You can also iterate with async for doc in items.find(query): to process documents one at a time.

Querying and indexes

Filters are dictionaries: {"price": {"$gte": 10, "$lt": 50}}, {"tags": "office"} (matches arrays that contain it), {"$or": [...]}. Updates use operators such as $set, $inc, $push and $unset. For anything that combines several steps, such as grouping and totals, use aggregate([...]) with a pipeline.

Queries on a field you filter or sort by need an index, or MongoDB scans the whole collection. Create indexes once at startup, which is idempotent:

Python
await db["items"].create_index("tags")
await db["items"].create_index([("price", 1)])
await db["users"].create_index("email", unique=True)   # a unique index enforces uniqueness

A duplicate insert into a unique index raises DuplicateKeyError (from pymongo.errors); catch it and answer 409 Conflict.

Production notes

  • Create the client once, in lifespan. A client per request throws away the connection pool.
  • Time limits: set serverSelectionTimeoutMS on the client so a missing database fails fast rather than hanging requests.
  • Validate in Pydantic, not only in the database. MongoDB itself accepts any shape unless you add JSON-schema validation to the collection.
  • No joins by default. Model data for how you read it: embed things that are read together, reference the rest by id, and use $lookup sparingly.
  • Transactions exist for replica sets and sharded clusters, but most single-document operations are already atomic.
  • Tests: swap the dependency for a mock client (as done here), or run a throw-away MongoDB container as described in Testing FastAPI.
  • To learn MongoDB itself, see the MongoDB tutorial.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Where should the MongoDB client be created in a FastAPI app?

    The client owns a connection pool and is designed to be shared. Creating one per request wastes connections.
  2. 2. Why does ObjectId cause trouble when returning documents?

    Pydantic and JSON do not know ObjectId. Converting it to str for output solves the problem.
  3. 3. What should happen when a URL contains a malformed id such as abc?

    ObjectId('abc') raises InvalidId. Validating in a dependency turns bad input into a clean client error.
  4. 4. What does matched_count == 0 after update_one tell you?

    matched_count counts documents that matched the filter; zero means nothing to update, a good place to raise 404.
  5. 5. Which statement about Motor and PyMongo is correct today?

    The official driver now offers async support directly, and the method names are nearly the same as Motor's.

Summary

  • Use AsyncMongoClient from PyMongo (or Motor's AsyncIOMotorClient); create it once in lifespan and keep it on app.state.
  • Give endpoints the collection through a dependency, which also makes testing easy.
  • Convert ObjectId to str for output with BeforeValidator(str) and validate incoming ids with ObjectId.is_valid.
  • CRUD maps to insert_one, find_one/find().to_list(), update_one with $set, and delete_one; check matched_count and deleted_count to detect missing documents.
  • Add indexes for fields you query, use range pagination for big collections, and catch DuplicateKeyError for unique indexes.

Next: see the wider picture in FastAPI vs Flask vs Django, or continue to Alembic migrations for the relational side.