Skip to content
Tutorials Center
My learning

Topics

Lesson 34 of 65 ยท Asynchronous Systems

Saga Pattern: Distributed Transactions

Learn the saga pattern for multi-service workflows: local transactions, compensating actions, choreography versus orchestration, idempotency and a runnable Python example.

Inside one database, a transaction is easy: either all changes commit or none do. Once data is spread across services (each with its own private database), a business operation such as "place an order" touches several of them: create the order, reserve stock, charge the card, schedule shipping. No single ACID transaction spans them. If the card charge fails after stock is reserved, something must put the stock back.

A saga is the standard answer. It breaks a distributed transaction into a sequence of local transactions, one per service. If a step fails, the saga runs compensating transactions to undo the steps that already succeeded.

Local steps and compensations

Each step has two parts: the action, and the compensation that semantically undoes it.

Step Action Compensation
1 Create order (pending) Cancel order
2 Reserve stock Release stock
3 Charge payment Refund payment
4 Arrange shipping Cancel shipment

Compensations are business-level undos, not database rollbacks. The original change was committed and might have been visible to others, so you cannot erase it; you issue a new action that cancels its effect (a refund, a restock). Some steps cannot be undone at all (an email already sent), so put such steps last, after everything that can fail.

flowchart LR
  A[Create order] --> B[Reserve stock] --> C[Charge payment] --> D[Ship]
  C -. fails .-> R2[Release stock]
  R2 --> R1[Cancel order]

If payment fails, compensate backward: release stock, cancel order.

Python A saga that rolls back with compensations

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Look at the final state: stock and wallet match what the successful order consumed, and the failed orders are marked cancelled. Between the steps and their undo, other requests could see the intermediate state, which brings us to the saga's biggest weakness.

Two ways to coordinate

Choreography. Each service listens for events and publishes its own. OrderCreated triggers stock reservation, which publishes StockReserved, which triggers payment, and so on. A failure publishes an event (PaymentFailed) that earlier services react to by compensating. Good for a few steps; hard to follow as it grows. See Event-Driven Architecture.

Orchestration. A saga orchestrator holds the workflow definition and sends commands to services in order, waits for replies and triggers compensations on failure. Its state (current step, outcomes) is persisted, so it survives restarts. This is the style of workflow engines such as Temporal, AWS Step Functions and Camunda. It centralizes the logic, which is easier to understand, test and monitor.

Choreography Orchestration
Control Distributed in events Central orchestrator
Coupling Services know event types Orchestrator knows all steps
Visibility Low; trace events High; one place shows progress
Risk Hidden cycles, hard changes Orchestrator becomes a bottleneck or a god service
Best for 2-4 simple steps Long, branching workflows

Sagas are not ACID

A saga gives you atomicity of outcome (eventually all steps done or all compensated) and durability, but not isolation. While a saga runs, other transactions can see its half-finished state: stock reserved but payment pending. This causes anomalies such as lost updates and dirty reads. Defenses ("countermeasures") include:

  • Semantic locks: mark the order PENDING so other flows know it is in progress.
  • Commutative updates: design operations so order does not matter (stock -= 2, not stock = 3).
  • Pessimistic view: reorder steps so the riskiest, least reversible action happens last.
  • Re-read before write: verify values before overwriting them.

Making sagas reliable

  • Idempotent steps and compensations: messages are delivered at least once and the orchestrator retries, so reserve and release must be safe to repeat. Use a saga ID or unique request key as the idempotency key.
  • Retry transient errors; compensate on business failures. "Network timeout" should retry; "insufficient funds" should compensate.
  • Compensations must not fail permanently. Retry them until they succeed, and alert a human if they cannot.
  • Persist saga state before and after each step so crash recovery knows where it stopped.
  • Timeouts: a saga stuck waiting for a reply needs a deadline and a decision (retry or compensate).
Python Idempotent compensation survives retries

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Saga versus two-phase commit

Two-phase commit (2PC) achieves atomicity across databases by locking resources until a coordinator decides. It gives true isolation, but it blocks if the coordinator fails, ties up locks across the network and is unsupported by most NoSQL stores and message brokers. Sagas trade isolation for availability and loose coupling, which is why they dominate in microservices.

Before building a saga, ask whether the operation really needs to span services. Often two steps that always change together belong in one service and one local transaction.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What is a compensating transaction?

    The original step is already committed, so you cannot roll it back. You issue a counteracting action such as a refund or restock.
  2. 2. Which ACID property do sagas NOT provide?

    Other transactions can observe a saga's intermediate state, so isolation is missing and must be handled with countermeasures.
  3. 3. Why must saga steps and compensations be idempotent?

    At-least-once delivery and orchestrator retries mean a step can run twice, so repeating it must be harmless.
  4. 4. Where should a step that cannot be undone, like sending an email, be placed?

    If it is last, you only run it once everything else has succeeded, so you never need to compensate it.

Summary

  • A saga splits a distributed transaction into local transactions, each with a compensation.
  • On failure, run compensations in reverse order; they are business-level undos, not rollbacks.
  • Choreography passes events between services; orchestration centralizes the flow in a coordinator.
  • Sagas lack isolation: use semantic locks, commutative updates and sensible step ordering.
  • Make steps idempotent, persist state, and prefer keeping related changes in one service.

Next: High Availability and Failover