Skip to content
Tutorials Center
My learning

Topics

Lesson 28 of 65 ยท Databases at Scale

CAP Theorem in Practice

Understand the CAP theorem and PACELC for system design interviews and real systems: consistency, availability, partitions, CP vs AP choices and a Python demo.

The CAP theorem is the most quoted and most misquoted idea in system design. It says that when a distributed data store is hit by a network partition, it must choose between consistency and availability. This lesson explains what the three letters really mean, why "pick two of three" is misleading, and how the extended PACELC model describes what happens when there is no failure.

The three letters

  • Consistency (C): here it means linearizability: every read gets the most recent write, as if there were one copy. It is not the "C" in ACID.
  • Availability (A): every request to a non-failing node gets a non-error response, with no guarantee it holds the latest data.
  • Partition tolerance (P): the system keeps working when messages between nodes are lost or delayed.

A partition splits your nodes into groups that cannot talk to each other. Networks do fail, so on a real multi-node system partitions will happen sooner or later. Partition tolerance is therefore not optional, and the true choice is: while the network is partitioned, do you stay consistent or stay available?

Why you cannot have both during a partition

Imagine two replicas, A and B, holding balance = 100. The link between them breaks. A client writes balance = 50 to A. Another client now asks B for the balance. B has two options:

  • Answer 100: B stays available but returns stale data. The system chose availability (AP).
  • Refuse, or wait: B keeps data correct but fails a request. The system chose consistency (CP).

There is no third option, because B cannot know about the write that A accepted.

flowchart LR
  subgraph Side1[Partition 1]
    A[(Node A: balance 50)]
  end
  subgraph Side2[Partition 2]
    B[(Node B: balance 100)]
  end
  A -. link broken .- B
  U1[Client 1] -->|write 50| A
  U2[Client 2] -->|read?| B

B must either return stale 100 (AP) or refuse to answer (CP).

Python The same partition, two behaviors

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

In the AP run, B happily answered with stale data, and the replicas reconciled after the heal. In the CP run, writes and reads failed while split, but data never disagreed.

What CP and AP look like in the wild

  • CP-leaning: systems built on consensus such as ZooKeeper, etcd, Spanner and HBase. Minority partitions stop serving. Used for coordination, locks, configuration and money.
  • AP-leaning: Cassandra and DynamoDB at low quorum settings, Riak, and many caches and DNS. They keep answering and reconcile later. Used for feeds, carts, sessions and metrics.

Many systems are tunable: Cassandra with QUORUM reads and writes behaves CP-ish, with ONE it behaves AP-ish. So label operations, not databases.

Common misunderstandings

  1. "Pick any two." Partitions are not a choice. A single-node database is "CA" only because it cannot partition; once it is distributed, P comes free.
  2. "CP means the system is down." Only the minority side of a partition refuses requests; the majority side keeps going.
  3. "AP means no consistency." AP systems are still eventually consistent after the partition heals, and they can offer session guarantees.
  4. Ignoring normal operation. Partitions are rare. Most of the time you care about latency versus consistency, which CAP does not describe.

PACELC: the other 99.9% of the time

PACELC extends CAP: if there is a Partition, choose A or C; Else (normal operation), choose Latency or Consistency. Replicating synchronously gives consistency but adds latency; replicating asynchronously is fast but may be stale.

System style During partition Else (normal) Example
PA/EL Stay available Favor low latency Dynamo-style, Cassandra defaults
PC/EC Favor consistency Favor consistency Spanner, HBase, most single-leader SQL setups
PA/EC Stay available Favor consistency Some MongoDB configurations
PC/EL Favor consistency Favor latency Some PNUTS-style systems
Python The latency price of consistency (EL vs EC)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

In an interview, state your choice per feature: "Checkout is CP: I would rather reject an order than oversell. The product page is AP: a slightly stale price is fine." Then mention the normal-case trade-off as PACELC.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. A partition separates two replicas. A CP system will:

    CP chooses consistency over availability: the side that cannot confirm the latest data refuses to answer.
  2. 2. Why is 'partition tolerance' not really optional?

    You cannot choose not to have network failures, so the real decision is between C and A when they occur.
  3. 3. What does the 'ELC' part of PACELC describe?

    PACELC: if Partition, A or C; Else, Latency or Consistency. The 'else' half covers normal operation.
  4. 4. Which feature would you design as AP?

    A like count can be briefly wrong with little harm, so favoring availability and low latency is the right call.

Summary

  • CAP: during a network partition you must choose between consistency (linearizable reads) and availability.
  • Partitions are unavoidable in distributed systems, so the real choice is CP or AP.
  • CP systems refuse requests they cannot guarantee correct; AP systems answer and reconcile later.
  • PACELC adds the normal-case trade-off: latency versus consistency.
  • Decide per feature by the cost of stale data, and say so explicitly.

Next: CQRS and Database per Service