Skip to content
Tutorials Center
My learning

Topics

Lesson 24 of 77 ยท Models and the ORM

Transactions and Concurrency

Keep Django data consistent with transaction.atomic, ATOMIC_REQUESTS and on_commit, avoid lost updates with F() and locking, and handle IntegrityError safely.

Imagine transferring money between two accounts. The code subtracts from one balance and adds to the other. If the server crashes between the two steps, money has vanished. Or imagine two visitors who load the same counter at the same moment: both read 41, both write 42, and one visit is lost. These are not exotic problems. They happen on every busy site, and they are why databases have transactions.

A transaction groups several statements into one unit that either happens completely (commit) or not at all (rollback). In this lesson you will use transaction.atomic(), see how Django's default autocommit mode behaves, run code only after a successful commit with on_commit, avoid lost updates with F() expressions and locking, and handle IntegrityError the right way. You should be comfortable with basic queries and with constraints.

The examples run on an in-memory SQLite database inside your browser. SQLite has no true concurrent writers and ignores row locks, so we simulate two competing requests with two Python objects. The patterns are exactly what you would use on PostgreSQL or MySQL; places where behavior differs are called out.

Autocommit and atomic()

By default Django runs in autocommit mode: every save(), create(), update() or delete() is its own tiny transaction and is committed immediately. That is fine for one statement. When several changes must succeed or fail together, wrap them in transaction.atomic(). It works as a context manager and as a decorator.

flowchart TD
    A["with transaction.atomic():"] --> B["BEGIN"]
    B --> C["run your queries"]
    C --> D{"exception<br/>escaped the block?"}
    D -->|no| E["COMMIT: all changes saved"]
    D -->|yes| F["ROLLBACK: all changes undone"]

An atomic block commits when it ends normally and rolls back when an exception leaves it.

Rollback is triggered by an exception that leaves the block. If you catch an exception inside the block and carry on, Django thinks everything went well. Nest blocks and the inner ones become savepoints: a failure in the inner block undoes only the inner work, while the outer transaction continues.

Python A bank transfer with atomic()

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Notice what each failure left behind. The simulated crash happened after Ana's balance was reduced, yet the balances stayed at their pre-transfer values, because the whole block was rolled back. The overdraft was stopped by the CheckConstraint, and again nothing changed. In the nested block, the ValueError undid only Ana's update, while Ben's update in the outer block was kept.

Tip: Keep atomic blocks short. A transaction holds locks until it ends, so slow work inside it (calling an HTTP API, sending email, resizing an image) makes other requests wait. Do the slow things before the block or, better, after the commit, as the next section shows.

ATOMIC_REQUESTS

Wrapping every view in atomic() yourself is repetitive. Set "ATOMIC_REQUESTS": True in the database entry of your DATABASES setting and Django wraps each request in a transaction: a normal response commits, an exception rolls back. Individual views can opt out with the @transaction.non_atomic_requests decorator. The setting is simple and safe, but it keeps a transaction (and its locks) open for the entire view, including template rendering, so very busy sites prefer explicit atomic() blocks around only the writes that belong together.

The next example uses the in-process test client to run two requests, one view that crashes after saving and one that succeeds, with the setting off and on.

Python ATOMIC_REQUESTS rolls back a crashing view

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

With the setting off, the half-finished note stays in the database. With it on, only the successful request leaves data behind. Note that ATOMIC_REQUESTS rolls back when the view raises an exception; a view that catches the problem and returns an error page normally still commits unless you roll back yourself.

on_commit: act only after the data is safe

Suppose a view creates an order and then queues a confirmation email. If the email goes out inside the transaction and the transaction later rolls back, the customer is told about an order that does not exist. Worse, a background worker could look up the order before it is committed and not find it.

transaction.on_commit(callback) solves this by registering a function that Django calls only after a successful commit of the outermost atomic block. If there is no transaction, the callback runs immediately. If the transaction rolls back, it is dropped. Use it for emails, task-queue jobs, cache invalidation and webhooks. (In tests built on TestCase, which wraps each test in a transaction that never commits, use captureOnCommitCallbacks to run them.)

Python Callbacks that wait for the commit

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Lost updates and F() expressions

The classic concurrency bug is read-modify-write: load a row into Python, change a value, save it back. Two requests doing this at the same time overwrite each other.

sequenceDiagram
    participant A as Request A
    participant DB as Database
    participant B as Request B
    A->>DB: read count (0)
    B->>DB: read count (0)
    A->>DB: write count = 1
    B->>DB: write count = 1
    Note over DB: two visits, but count is 1

Both requests computed from the stale value 0, so the second write erases the first.

There are three standard cures:

  1. Let the database do the arithmetic. update(count=F("count") + 1) becomes UPDATE ... SET count = count + 1, which the database applies to the current value atomically. This is the best fix for counters, stock levels and balances.
  2. Optimistic locking. Keep a version column. Update with filter(pk=..., version=seen) and check how many rows changed. Zero means somebody else got there first, so reload and retry. It needs no locks and suits low-conflict data.
  3. Pessimistic locking. select_for_update() issues SELECT ... FOR UPDATE, which locks the selected rows until the transaction ends, so the second request waits its turn. It must run inside an atomic() block. Options include nowait=True (raise instead of waiting) and skip_locked=True (skip locked rows, handy for job queues). SQLite ignores it, and PostgreSQL, MySQL and Oracle honor it.
Python Lost update, F(), versions and select_for_update

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The F() version reached 2 because each statement was applied to the current stored value. After save() with an F() expression, the Python attribute holds the expression rather than a number, so call refresh_from_db() before reading it. The last lines show the optimistic check: the second attempt used an outdated version and was refused.

On PostgreSQL the locking version looks like this (read-only, since the live runner uses SQLite):

Python
with transaction.atomic():
    account = Account.objects.select_for_update().get(pk=account_id)
    account.balance -= amount      # nobody else can change this row until we commit
    account.save()

Handling IntegrityError

A unique or check constraint tells you when two requests (or one buggy request) tried to write conflicting data. The database raises IntegrityError, and you must handle it with care: on PostgreSQL, after a database error inside a transaction, no further query works until you roll back to a savepoint, and Django raises TransactionManagementError if you try. The remedy is to wrap just the risky statement in its own atomic() block inside a try, so the failure rolls back only to that savepoint.

Python
try:
    with transaction.atomic():
        Subscriber.objects.create(email=email)
except IntegrityError:
    ...   # the surrounding transaction is still healthy

get_or_create() and update_or_create() use this pattern internally, which makes them safe against two simultaneous "create if missing" requests, provided the lookup fields are protected by a unique constraint.

Python Catching IntegrityError without breaking the transaction

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Which tool when?

Problem Tool
Several writes must succeed together transaction.atomic()
Every view should be all-or-nothing ATOMIC_REQUESTS
Send email or queue a job after saving transaction.on_commit()
Counter, balance or stock updates F() expressions
Rare conflicts on long edit forms version column (optimistic)
Frequent conflicts on hot rows select_for_update() (pessimistic)
Duplicate inserts unique constraint + IntegrityError handling or get_or_create()

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Inside a transaction.atomic() block you catch an exception with try/except and keep going. What does Django do at the end of the block?

    atomic() rolls back only when an exception propagates out of the block. A caught exception looks like normal completion.
  2. 2. Why does transaction.on_commit() exist?

    Side effects such as emails or queued jobs must not happen for data that gets rolled back, and workers should find the committed rows. on_commit delays the callback until the outermost transaction commits.
  3. 3. Which line increments a counter safely when many requests run at once?

    F('count') + 1 is evaluated by the database against the current stored value. The other versions compute from a value that may already be stale.
  4. 4. You catch an IntegrityError inside an outer atomic block on PostgreSQL and then run another query. What is the safe way to write it?

    A database error poisons the current transaction on PostgreSQL. An inner atomic() creates a savepoint, and the failure rolls back only to it, leaving the outer transaction usable.

Summary

  • Django autocommits each statement; transaction.atomic() groups statements so they commit or roll back together.
  • Rollback happens when an exception leaves the block; nested blocks are savepoints.
  • ATOMIC_REQUESTS = True wraps each request in a transaction; non_atomic_requests opts out.
  • transaction.on_commit() runs side effects such as emails and jobs only after a successful commit.
  • Prevent lost updates with F() expressions, optimistic version checks, or select_for_update() (ignored by SQLite).
  • Wrap risky inserts in their own atomic() when catching IntegrityError, or use get_or_create().

Next: Raw SQL, Database Functions and JSON Fields.