Skip to content
Tutorials Center
My learning

Topics

Lesson 13 of 30 ยท Cassandra in Practice

Lightweight transactions, batches and counters

Learn when Cassandra's IF NOT EXISTS (lightweight transactions) is worth its cost, how logged and unlogged batches really work, and where counters bite.

Cassandra is built around fast, independent writes, so it does not give you the multi-row ACID transactions of a relational database. It does give you a few tools that look similar at first glance: lightweight transactions (compare-and-set writes), batches, and counters. Each one is easy to misuse, and most "Cassandra is slow" stories start with one of them. This lesson explains what each tool guarantees, what it costs and when to pick something else.

The real Cassandra cannot run on this site, so every CQL block below is read-only: it shows the real syntax but was not executed here. The JavaScript stand-ins are small models of the ideas (a compare-and-set, a batch log, a lost reply). They are not Cassandra's code, and they leave out replication, timestamps and most failure modes.

The problem: check, then write

Suppose usernames must be unique. The obvious approach is to read first and write second:

SQL
SELECT username FROM app.users_by_name WHERE username = 'ada';
-- no row, so the name looks free ...
INSERT INTO app.users_by_name (username, user_id) VALUES ('ada', ?);

Between the read and the write another client can run the same two steps. Cassandra writes are "last write wins", so both inserts succeed and the second silently overwrites the first. Both users believe they own the name.

JS Check-then-write loses a race

Press Run (or Ctrl + Enter) to see the console output here.

No output.

User A was told "success" and then lost the name without any error. We need a write that checks and writes as one atomic step.

Lightweight transactions

A lightweight transaction (LWT) adds an IF condition to a write. Cassandra evaluates the condition and applies the write as one atomic step for that partition:

SQL
INSERT INTO app.users_by_name (username, user_id)
VALUES ('ada', 7f1c9a52-0d3e-4a8e-9c1d-2b6f5e4a1c10)
IF NOT EXISTS;

UPDATE app.accounts
SET    email = '[email protected]'
WHERE  account_id = 42
IF     email = '[email protected]';

The result has an extra column, [applied], that is true when the write happened. When it is false, the row also returns the current values so you can see why:

TEXT
 [applied] | username | user_id
-----------+----------+----------------------------------------
     False |      ada | 3a0f5d1e-...

Conditions are only checked within a single partition, and a condition write may not be mixed with ordinary writes in a way that spans partitions.

The price: Paxos

Plain writes need one round trip to the replicas. An LWT runs a consensus protocol (Paxos) between the replicas of the partition so that concurrent conditional writes are put in one agreed order. The classic protocol needs several round trips: prepare, read the current value, propose, and commit. Newer Cassandra versions have an improved Paxos implementation with fewer round trips, but an LWT is still several times slower than a normal write, and contention on one partition makes it worse because competing proposers retry.

LWTs also use a second consistency setting, the serial consistency level (SERIAL or LOCAL_SERIAL), that controls the consensus step; the ordinary consistency level then applies to the commit. For one datacenter you normally pick LOCAL_SERIAL.

JS A compare-and-set closes the race

Press Run (or Ctrl + Enter) to see the console output here.

No output.

The first client wins and the second is told applied: false, together with the current owner. The cost is the extra round trips, which is why you use LWTs for rare, important decisions (claiming a username, creating an account, leader election) and never for every write.

Batches

A batch groups several statements into one request:

SQL
BEGIN BATCH
  INSERT INTO shop.orders_by_user (user_id, order_id, total) VALUES (?, ?, ?);
  INSERT INTO shop.orders_by_id   (order_id, user_id, total) VALUES (?, ?, ?);
APPLY BATCH;

There are two kinds:

  • A logged batch (the default) first writes the whole batch to a batch log on other nodes, then runs the statements. If the coordinator dies halfway, a surviving node replays the batch, so the statements are eventually all applied. This is atomicity across partitions, and it is the main legitimate use: keeping several denormalized query tables in step (see Query-first data modeling).
  • An unlogged batch (BEGIN UNLOGGED BATCH) skips the batch log. There is no cross-partition guarantee; it is only reasonable when every statement targets the same partition.

Be clear about what a batch does not give you: there is no isolation (readers can see half of a batch while it is being applied), and no rollback. It is also not a bulk-loading tool. A batch with statements for many partitions makes one coordinator do all the work, and logged batches add the batch-log writes on top.

flowchart LR
    C["Client"] --> Co["Coordinator"]
    Co --> BL["Batch log<br/>on two other nodes"]
    BL --> P1["Partition 1 replicas"]
    BL --> P2["Partition 2 replicas"]
    P2 -. "after success" .-> RM["Batch log entry removed"]

A logged batch is stored first, applied second, and removed afterwards, so a crash in the middle can be repaired by replay.

JS Crash in the middle of a batch

Press Run (or Ctrl + Enter) to see the console output here.

No output.

After the crash the unlogged run has order-1 in only one table, while the logged run was repaired from the batch log. In real Cassandra the replay happens later, on another node, so the two tables can disagree for a short time before they agree.

Why big batches are an anti-pattern

Cassandra logs a warning when a batch is larger than batch_size_warn_threshold (5 KiB by default in recent versions) and rejects it above batch_size_fail_threshold (50 KiB). These limits exist because a large multi-partition batch stresses the coordinator and the batch log. To load many rows quickly, send many single-row asynchronous writes (a driver does this well, see Drivers, nodetool and production pitfalls), or group rows by partition and batch only rows that share one.

Counters

Counters were introduced in Data types, collections and UDTs. Two more points matter in practice. A counter update is a read-modify-write on the replica, so it is more expensive than an ordinary write. And counter updates cannot be mixed into a normal batch (a BEGIN COUNTER BATCH exists for counters only), and they are not idempotent: a retried increment can double count. Use them for approximate numbers, and store events as rows when you need exact totals.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why can a read followed by a plain INSERT not guarantee a unique username?

    The check and the write are two separate operations. A concurrent client can pass the same check, and Cassandra then keeps whichever write is newer.
  2. 2. What is the main cost of a lightweight transaction?

    Paxos orders competing conditional writes by exchanging several messages with the replicas of the partition. That is the price of the atomic check.
  3. 3. What does a logged batch guarantee?

    The batch log lets another node replay the batch. There is no isolation and no rollback, and batches spanning many partitions are slower, not faster.
  4. 4. When is an unlogged batch reasonable?

    Statements for one partition are applied together on the same replicas, so the batch log adds nothing. For many partitions it only moves work to one coordinator.

Summary

  • Check-then-write is racy in Cassandra; IF NOT EXISTS and IF column = value make the check and write one atomic step within a partition.
  • LWTs use Paxos and cost several round trips, so reserve them for rare decisions such as claiming a unique name.
  • A logged batch makes several tables or partitions eventually consistent with each other; it gives no isolation or rollback.
  • Batches are not a performance feature. Keep them small and use them to keep denormalized tables in step.
  • Counters are cheap-ish but not idempotent; store events when the number must be exact.
  • The stand-ins model the ideas only; the CQL was not run here.

Next: Secondary indexes, SAI and materialized views.