Skip to content
Tutorials Center
My learning

Topics

Lesson 13 of 37 ยท Key-Value and Config

Check-and-Set, Sessions and Distributed Locks

Update Consul KV safely with check-and-set, use sessions with TTLs and behaviors, acquire and release locks, run consul lock, and build leader election with a stale-lock guard.

Two programs read the same key, change it, and write it back. The second write silently erases the first. This lost update is the classic bug of shared configuration, and Consul gives you two cures: check-and-set (optimistic concurrency) for individual updates, and sessions with locks (pessimistic exclusion) for "only one of us may do this".

The API calls and commands are read-only; the stand-ins model both mechanisms so you can see the retries and the lock timeline.

Check-and-set (CAS)

Every key has a ModifyIndex. A CAS write says "store this value only if the key has not changed since index N":

Terminal
$ curl "http://127.0.0.1:8500/v1/kv/app/counter?cas=137" --request PUT --data '8'
true

The API replies true if the write happened and false if someone else got there first (their write changed ModifyIndex). With cas=0 the write succeeds only when the key does not exist yet, which makes it a safe "create if absent". The CLI form:

Terminal
$ consul kv put -cas -modify-index=137 app/counter 8

The standard loop is: read the value and its ModifyIndex; compute the new value; write with cas=<that index>; if it returned false, start over.

Sessions

A session is a handle that represents a client's liveness. Create one, attach it to keys to take locks, and Consul releases the locks automatically when the session ends.

Terminal
$ curl --request PUT http://127.0.0.1:8500/v1/session/create --data '{
  "Name": "job-runner",
  "TTL": "30s",
  "Behavior": "release",
  "LockDelay": "15s"
}'
{"ID": "adf4238a-882b-9ddc-4a9d-5b6758e4159e"}
Field Meaning
Name Human label
TTL The session is invalidated unless renewed within this time (10s to 86400s). The effective timeout can be up to twice the TTL
Behavior release (default) frees locks when the session ends; delete deletes the locked keys
LockDelay After a session ends, its locks cannot be acquired by others for this long (default 15s), protecting against split-brain
Checks Health checks that must stay passing, otherwise the session is invalidated

Keep a TTL session alive with PUT /v1/session/renew/<id>, at about half the TTL. Delete it explicitly with PUT /v1/session/destroy/<id>.

Taking a lock

Locks are KV writes with acquire:

Terminal
$ curl --request PUT "http://127.0.0.1:8500/v1/kv/service/web/leader?acquire=$SESSION" --data 'node-a'
true

The first client to acquire gets true; others get false and can wait with a blocking query on the key (see watches). Consul records the holder in the key's Session field and bumps LockIndex. Release with ?release=<session>, or let the session expire.

consul lock

For simple cases the CLI takes care of everything: it creates a session, acquires the key, runs your command, renews the session, and releases when the command ends.

Terminal
$ consul lock service/nightly-backup ./backup.sh

Only one machine at a time can run ./backup.sh; the others wait. If the lock is lost (session invalidated), the child process is terminated, so two copies never run together. Use -n=2 to allow two holders, and -try=10s to give up instead of waiting forever.

Leader election pattern

sequenceDiagram
  participant A as Instance A
  participant K as KV key service/web/leader
  participant B as Instance B
  A->>K: acquire (session A)
  K-->>A: true (A is leader)
  B->>K: acquire (session B)
  K-->>B: false
  B->>K: blocking query on the key
  Note over A: A crashes, session A expires
  K-->>B: key released, index changed
  B->>K: acquire (session B)
  K-->>B: true (B is leader)

Everyone competes for the same key; the winner is leader until its session ends.

A lock is only advisory, and a paused process can believe it is still leader after its session expired (for example during a long garbage collection pause). Your leader must watch for session loss and stop work immediately. LockDelay reduces the window but cannot close it; use fencing tokens (for example the key's LockIndex) when a stale leader could do harm.

Stand-in 1: a CAS retry loop

Two clients try to increment the same counter. Without CAS one update vanishes; with CAS the loser re-reads and tries again. The model uses a tiny store with ModifyIndex.

Python Stand-in: check-and-set

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Stand-in 2: session locks with TTL and lock-delay

A simulated clock steps through a timeline: node A takes the lock, stops renewing (it crashes), its session expires, and B must wait out the lock delay before acquiring. Times are in seconds.

Python Stand-in: lock timeline

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does a PUT with ?cas=0 mean?

    cas=0 succeeds only when the key is absent, making it a safe create-if-missing.
  2. 2. A CAS write returns false. What should the client do?

    False means another writer changed the key; the standard loop is read, compute, CAS, retry.
  3. 3. What does LockDelay protect against?

    The delay gives a possibly stale holder time to notice its loss before someone else takes over.
  4. 4. What does consul lock do when it loses its session?

    The child is stopped when the lock is lost, preserving mutual exclusion.

Summary

  • CAS (?cas=<ModifyIndex>) prevents lost updates; false means retry with a fresh read.
  • Sessions represent liveness: TTL, behavior (release or delete), lock delay and optional health checks.
  • Acquire locks with ?acquire=<session>; release with ?release= or let the session end.
  • consul lock wraps session, lock and child process for simple exclusive jobs.
  • Locks are advisory: stop work when the session is lost and use fencing when it matters.

Next: Watches and Blocking Queries