Skip to content
Tutorials Center
My learning

Topics

Lesson 24 of 36 ยท State and Concurrency

Refs and Software Transactional Memory

Coordinate several pieces of state safely with Clojure refs and software transactional memory: dosync, alter, commute, ref-set, ensure, retries and the bank transfer example.

An atom is perfect for one independent piece of state. But some changes involve several pieces at once: moving money from one account to another must subtract from one and add to the other, and no other thread should ever see the money missing or doubled. Locks can do this, but locks bring deadlocks and forgotten-lock bugs. Clojure offers software transactional memory (STM) instead, using the ref reference type.

Refs and dosync

A ref holds a value like an atom does, but it can be changed only inside a transaction, opened with dosync.

CLOJURE
(def alice (ref 100))
(def bob   (ref 50))

(defn transfer [from to amount]
  (dosync
    (alter from - amount)
    (alter to   + amount)))

(transfer alice bob 30)

[@alice @bob]   ;=> [70 80]

Everything inside dosync is one atomic unit: other threads see either both changes or neither, and a failed transfer leaves both balances untouched. Reading a ref outside a transaction with @ is allowed and gives the latest committed value.

Attempting (alter alice inc) outside dosync throws IllegalStateException: No transaction running.

How transactions work

The STM uses multiversion concurrency control (MVCC), the same idea as many databases. Each transaction works with a consistent snapshot of the world taken when it starts. When it finishes, the system checks whether any ref the transaction changed was also changed by someone else. If not, the transaction commits; if there is a conflict, it rolls back and retries from the start.

flowchart TD
  S["dosync starts: take snapshot"] --> W["run body against snapshot"]
  W --> C{"any ref changed by another transaction?"}
  C -- no --> OK["commit all changes atomically"]
  C -- yes --> R["discard work and retry body"]
  R --> W

Readers never block writers and vice versa, because each transaction has its own snapshot.

Because the body may run more than once, the same rule as for swap! applies: no side effects in a transaction. Do not print, send email, write to a file or update an atom inside dosync. If you need an effect after a successful commit, use an agent: send calls made inside a transaction are held until the transaction commits, and are dropped if it retries (see agents).

alter, ref-set and commute

Three functions change a ref:

  • (alter r f & args) sets the ref to (f current args...). It is the workhorse and conflicts when the value was changed concurrently.
  • (ref-set r v) replaces the value outright. Use it when you are initializing or resetting, not when the new value depends on the old.
  • (commute r f & args) is for operations where order does not matter, such as incrementing a counter or adding to a set.

commute is less strict. Another transaction changing the same ref does not force a retry, because the function can be re-applied at commit time. That makes it much faster under contention, but you must pick functions that are commutative (the result is the same in any order):

CLOJURE
(def hits (ref 0))
(dosync (commute hits inc))         ; fine: inc commutes
(dosync (commute hits #(* % 2)))    ; WRONG: order matters, doubling is not commutative with inc

Inside the transaction commute returns an in-transaction value, but the final stored value is recomputed at commit from the most recent committed state.

ensure and write skew

Reading a ref in a transaction does not stop another transaction from changing it. If your decision depends on a value you read but do not change, you can fall into write skew: two transactions each read a shared ref, each decide it is safe to proceed, and together break an invariant.

(ensure r) reads the ref and protects it, so the transaction retries if anyone changes it before commit.

CLOJURE
(def on-call (ref #{"ana" "ben"}))  ; invariant: at least one person on call

(defn go-off-call [person]
  (dosync
    (when (> (count (ensure on-call)) 1)   ; protect the read
      (alter on-call disj person))))

Without ensure, both could leave simultaneously because each saw two people. Using alter on the same ref (instead of ensure) also protects it.

Validators, watches and ref history

Refs support validators and watches like atoms. A validator on a ref makes a transaction that would break an invariant fail and roll back as a whole:

CLOJURE
(def acct (ref 100 :validator #(>= % 0)))
(dosync (alter acct - 500))   ; throws; no partial effect

Atoms, refs, agents: choosing

Need Use
One independent value, synchronous atom
Several values changed together, synchronous ref + dosync
Asynchronous, serialized updates; side effects agent

Refs are used less in practice than atoms, because most programs get by with one atom holding one map. When you do need coordination, STM is clear and deadlock-free: you state what must change together, and the system handles the rest. A common compromise is to put all related data in a single atom and update it with one swap!, which is already atomic for that whole value.

Try it: a transactional transfer in JavaScript

The stand-in below does not implement MVCC; it models the effect on a single thread: stage the changes, check an invariant, then commit all of them or none.

JS All or nothing transfer

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Try it: why retries need pure functions

This stand-in simulates a conflict that forces a retry and counts how often the transaction body ran, showing why side effects do not belong there.

JS Retries run the body again

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What happens if you call (alter r inc) outside a dosync?

    Refs can only be changed inside a transaction; otherwise Clojure throws IllegalStateException.
  2. 2. Why shouldn't you print inside a dosync?

    On conflict the transaction is retried, so side effects would repeat. Use agents for effects after commit.
  3. 3. Which operation is a good fit for commute?

    Increment commutes: any order gives the same total. Commute avoids needless retries for such operations.
  4. 4. What does ensure do?

    ensure prevents write skew by making the transaction retry if the protected ref is changed by someone else.

Summary

  • Refs change only inside dosync transactions, which are atomic, isolated and retried on conflict.
  • Use alter (default), ref-set (replace) and commute (order-independent updates).
  • Transaction bodies must be free of side effects.
  • ensure protects values you read but do not write, avoiding write skew.
  • Use an atom for independent state, refs when several values must change together.

Next: Agents, Futures and Promises