Skip to content
Tutorials Center
My learning

Topics

Lesson 9 of 30 ยท How Cassandra Works Inside

The write path: commit log, memtable, SSTable

See why Cassandra writes are fast: append to a commit log, update an in-memory memtable, flush immutable SSTables. Includes a runnable LSM stand-in.

Cassandra can absorb enormous write loads, and the reason is not magic hardware. It is a deliberate storage design: a write never searches for the old value and never edits a file in place. Instead, a write is recorded in two cheap places, one on disk that is only ever appended to and one in memory, and the heavy lifting happens later in the background. This design is called a log-structured merge tree (LSM tree), and it explains many of Cassandra's quirks, from why updates and inserts are the same thing to why deletes are strange.

In this lesson you follow a single write through one node, then build a miniature version in JavaScript. The stand-in imitates the structure: it is not Cassandra's code, it knows nothing about real file formats, and its counts are not benchmarks. Real Cassandra is shown in the diagram and described in words, because a database server cannot run in a web page. Before this lesson, it helps to know what a partition is and how the ring places data on nodes.

One write, step by step

When a client sends INSERT or UPDATE, one node acts as the coordinator and forwards the write to the replicas that own the partition (more on that in Replication and tunable consistency). On each replica the write goes through the same steps:

flowchart LR
    C["Write arrives<br/>at a replica"] --> L["1. Append to<br/>commit log (disk)"]
    C --> M["2. Update<br/>memtable (memory)"]
    L --> A["3. Acknowledge<br/>to coordinator"]
    M --> A
    M -- "memtable full" --> F["4. Flush to a new<br/>SSTable (disk)"]
    F --> D["Commit log segments<br/>can be discarded"]

A write is durable once it is in the commit log and visible once it is in the memtable; the SSTable comes later.

  1. Commit log. The write is appended to a sequential log file on disk. Appending to the end of a file is the cheapest thing a disk can do, even a spinning one. The commit log exists only for crash recovery: nothing reads it during normal operation.
  2. Memtable. The same write is applied to an in-memory sorted structure, one per table. Reads can see it immediately.
  3. Acknowledgement. The replica tells the coordinator it is done. At no point did it read an existing row from disk.
  4. Flush. When the memtable grows past a limit (or the commit log space fills up, or an operator runs nodetool flush), its content is written out as a new SSTable (sorted string table). The flushed commit log segments are no longer needed and are recycled.

What an SSTable is

An SSTable is a set of files on disk holding rows sorted by partition key and, within a partition, by clustering key. The crucial property is that it is immutable: once written, it is never modified. Alongside the data file Cassandra writes helper files, such as an index that tells where each partition starts and a bloom filter that quickly says "this key is definitely not here" (you will meet it in The read path, bloom filters and compaction).

Immutability has big consequences:

  • No locking or in-place edits. Many readers can use a file safely while writes continue elsewhere.
  • Updates are new data. Changing a value writes a newer version of the cell into the memtable and eventually into a newer SSTable. The old version stays in the older SSTable until compaction merges them. Every cell carries a write timestamp, and the newest timestamp wins.
  • Deletes are data too. Removing a value writes a marker called a tombstone (see Deletes, TTL and tombstones).

The write is cheap because the expensive work, sorting, merging and reorganizing, is postponed and done in bulk.

Durability and the commit log

The memtable lives in RAM, so a crash would erase it. The commit log is what saves you: when the node restarts, it replays the log and rebuilds the memtables that had not been flushed yet. By default the commit log is synced to disk periodically (every few seconds) instead of on every write, so after a power loss a single node could lose its last moments of writes. That trade-off is acceptable because the same write also exists on other replicas. If you need a stricter setting, Cassandra offers a mode that syncs in batches before acknowledging, at the cost of latency.

Stand-in 1: a miniature LSM store

The class below has a commit log, a memtable, immutable sorted "SSTables", a write timestamp on every value, and reads that check newest data first. The memtable flushes when it holds three keys. Read the output and note how an update to "b" ends up in two places.

JS A tiny log-structured store

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Notice three things. The first three writes filled the memtable and produced SSTable 1, where the rows are sorted by key rather than by arrival order. The second write to "b" did not touch SSTable 1; it created a newer value that now sits in SSTable 2, and a read finds the newest version first. And reading "a" had to look through two SSTables, which is the price paid for cheap writes. Reads get the attention in the next lesson.

Stand-in 2: surviving a crash

Now simulate a crash. The memtable is lost, and recovery replays the commit log. The program also runs the same crash on a node without a commit log to show what the log protects against.

JS Crash, replay, and what the log buys you

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Without the log, the unflushed write disappears. With it, replay restores the write. The flushed write survives either way, because it was already in an immutable file.

Why writes are fast, and what it costs

Collect the reasons:

  • Sequential I/O only. Appends to the commit log and one big write per flush; no random seeks.
  • No read before write. A normal insert or update does not look at old data. (Exceptions exist, such as lightweight transactions and some list operations, and they are slower for exactly that reason.)
  • Memory first. The caller waits for a memory update and a log append, not for sorting a file.
  • Everything is an upsert. An INSERT for an existing primary key simply writes a newer version.

The costs appear elsewhere. Reads may have to consult several SSTables and merge them. Disk space holds old versions until compaction cleans up. A flood of tiny flushes would leave too many files, so Cassandra runs compaction in the background, which is exactly what the next lesson covers.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What is the main purpose of the commit log?

    The commit log is an append-only record used for crash recovery. Normal reads never use it.
  2. 2. Why can Cassandra accept writes so quickly?

    The expensive work, sorting and merging, is postponed to flushes and compaction, so the write path touches only a log append and memory.
  3. 3. You update a value that already sits in an SSTable. What happens to the old SSTable?

    SSTables are immutable. The newest timestamp wins when versions are merged, and compaction later removes the superseded data.
  4. 4. When is a memtable flushed to an SSTable?

    Flushes are triggered by memtable size, commit log pressure or an explicit nodetool flush, not by individual reads or writes.

Summary

  • A write goes to an append-only commit log and an in-memory memtable; no old data is read.
  • Full memtables are flushed into immutable, sorted SSTables, and the used commit log segments are recycled.
  • Updates and deletes create newer data (with timestamps or tombstones); older versions disappear later during compaction.
  • The commit log makes writes durable across crashes, and replicas on other nodes add more protection.
  • The JavaScript stand-ins show the structure only; real Cassandra's files, thresholds and speed differ.

Next: The read path, bloom filters and compaction.