Skip to content
Tutorials Center
My learning

Topics

Lesson 18 of 36 ยท Consensus and Coordination

CAP, PACELC and Real Trade-offs

What the CAP theorem really says, why partitions are not optional, how PACELC adds the latency trade-off, and a CP vs AP partition simulation in Python.

"Pick two of three: consistency, availability, partition tolerance." You have probably seen this slogan, and it is one of the most misunderstood statements in the field. Read literally it suggests a menu from which you choose a combination. In reality it describes one narrow situation, and the real design decisions are broader than that.

In this lesson you will learn what the CAP theorem actually claims, why "partition tolerance" is not something you can skip, what PACELC adds for the time when nothing is broken, and how to think about real systems without sticking labels on them blindly. A small simulation lets you see the trade-off with your own eyes.

What the three letters mean

CAP uses very specific definitions, narrower than everyday use of the words:

  • Consistency (C) means linearizability (see consistency models): every read returns the latest completed write, as if there were a single copy of the data. It is not the "C" in ACID.
  • Availability (A) means that every request to a non-failed node receives a non-error response, with no promise that it is up to date. It is not "99.99% uptime".
  • Partition tolerance (P) means the system keeps working even when the network drops messages between groups of nodes.

What the theorem says

Take two replicas of a value, on two sides of a network partition that stops them from talking. A client writes x = 2 on one side. Another client reads x on the other side. The second node has two options:

  1. Return its old value x = 1. It stays available, but the read is not consistent: it missed a completed write.
  2. Refuse (or wait until the partition heals). It stays consistent, but it is not available.

There is no third option. So: during a partition, you must choose between consistency and availability. That is the whole theorem.

CAP in one sentence

Partition happens Nodes cannot reach each other.
Choose CP Refuse some requests to stay correct.
Choose AP Answer everything, maybe with stale or divergent data.

Without a partition, you can have both C and A.

1 / 2

Partitions are not optional

Because partitions are not something you pick, "CA" is not a real category for a distributed system. Networks fail: cables are cut, switches reboot, a long GC pause makes a node look dead, a cloud zone loses connectivity (see the fallacies of distributed computing). A single-node database is "CA" only because it is not distributed. Once your data lives on several machines, P is a fact and C versus A is the choice you make when it strikes.

Common misreadings to avoid:

  • "My system is AP or CP as a whole." The choice can differ per operation, per data type and per configuration. A shopping cart may be AP while payment is CP.
  • "CAP says you can never have consistency and availability." It only says you cannot have both while partitioned. Most of the time there is no partition.
  • "AP systems have no consistency." They usually offer weaker models such as eventual or causal consistency.

Simulating a partition

We build a tiny three-node store. A partition separates node C from nodes A and B. The CP variant requires a majority (2 of 3) to accept reads and writes, so the isolated node refuses them. The AP variant lets every node answer from its local copy and merges later with last-write-wins.

Python CP versus AP under a partition

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

In CP mode the isolated node C says no to everything, but every answer given is correct, and no write is ever lost. In AP mode C happily accepted v3, answered reads with stale data while the partition lasted, and when the partition healed, both writes carried version 2, so the merge broke the tie by comparing the data, kept v3 and silently dropped v2, even though a client had been told that v2 succeeded. That loss is the cost of availability; a CRDT or merge function can reduce it.

PACELC: the rest of the time

CAP says nothing about normal operation, yet that is 99.9% of the time. Daniel Abadi's PACELC formulation completes the picture:

If there is a Partition, choose between Availability and Consistency; Else (no partition), choose between Latency and Consistency.

Even without failures, keeping replicas strongly consistent costs time: a write must wait for replicas to acknowledge it, and a consistent read may need to contact a leader or a quorum. If a replica is on another continent, that is tens or hundreds of milliseconds. Systems that answer from the nearest replica are faster but may serve stale data. So the trade-off between latency and consistency exists every day, not only during outages.

Here is the "else" half in numbers. A client sits in Europe; the replicas are in Europe (10 ms away), the US (90 ms) and Asia (200 ms). We compare an answer from the nearest replica with a majority round (wait for the 2nd-fastest of three replicas).

Python The latency price of consistency

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

No partition is involved at all, yet the consistent choice is nine times slower.

Classifying real systems

The labels are shorthand for default behavior, and many databases can be configured both ways (see quorums):

System style During a partition Normally PACELC
Single-leader with synchronous replication and majority writes (e.g. systems built on Raft) Minority side stops Pays round trips for consistency PC/EC
Dynamo-style stores with low quorums Stay available, may diverge Fast local answers PA/EL
Dynamo-style stores with R + W > N Can fail some requests Slower reads and writes PC/EC (as configured)
Single-leader with asynchronous followers read from locally Often stay available Fast, stale reads PA/EL

Treat any table like this as a starting point and read each system's documentation for exact guarantees. Marketing pages often claim more than the system delivers under failure.

Using this in design

  • Ask per feature: what is worse here, an error or a stale answer? A "like" count can be stale; a bank balance probably cannot.
  • Decide what happens to the minority side of a partition, and tell users.
  • Measure latency to the nearest replica; strong consistency across regions is expensive.
  • Don't hide the decision behind slogans: write it down per data set.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does the CAP theorem say?

    The forced choice applies while nodes cannot communicate. Without a partition you can have both.
  2. 2. Why is 'CA' not a practical option for a distributed system?

    You do not choose whether partitions happen. P is forced on you, so the real choice is between C and A when one occurs.
  3. 3. What does PACELC add to CAP?

    Even in normal operation, stronger consistency costs coordination time, so there is a latency-versus-consistency trade-off.
  4. 4. In the CP simulation, why does the isolated node reject requests?

    A CP design refuses service when it cannot confirm correctness; the minority side stops.

Summary

  • CAP's consistency means linearizability and its availability means every live node answers; the theorem applies during a partition.
  • Partitions are a fact of networks, so the choice is between CP (refuse some requests) and AP (answer, maybe stale or divergent).
  • PACELC adds: otherwise you choose between latency and consistency.
  • Systems are configurable and the choice can differ per operation; judge them by the behavior you need.

Next: Failure Detectors and Heartbeats.