Skip to content
Tutorials Center
My learning

Topics

Lesson 28 of 37 ยท Deno KV and Data

Deno KV Atomic Operations, Queues and Watch

Make safe multi-key updates with Deno KV atomic operations, handle races with versionstamp checks and retries, count with sum, run background jobs with queues and react to changes with watch.

A key-value store that only does get and set is fine for a single user and a single request. A real server handles many requests at the same moment, and two of them can read the same value, change it and write back, so one change silently disappears. This is the classic lost update problem. Deno KV solves it with atomic operations: a group of reads-as-checks and writes that either all happen or none do.

KV also has two features that turn it from a store into an application backbone: queues (run work later, with retries) and watch (get notified when a key changes). This lesson explains all three.

Deno KV exists only inside Deno, so Deno.openKv code is read-only. The runnable examples use MiniKv, an in-memory stand-in with the same method names (atomic().check().set().commit(), enqueue, listenQueue, watch) as introduced in Deno KV Basics. Replace openKv() by Deno.openKv() for the real thing. On some Deno versions KV, queues or cron may need --unstable-kv (or an unstable entry in deno.json).

Atomic operations

Start a transaction with kv.atomic(), chain checks and mutations, and commit():

TypeScript
const result = await kv.atomic()
  .check({ key: ["balance", "ada"], versionstamp: current.versionstamp })
  .set(["balance", "ada"], current.value - 50)
  .set(["balance", "grace"], other.value + 50)
  .commit();

if (!result.ok) {
  // a check failed: someone changed a key since we read it
}
  • check({ key, versionstamp }) says "only continue if this key still has the versionstamp I saw". A versionstamp of null means "this key must not exist yet".
  • Mutations are set, delete, sum, min, max and mutate.
  • commit() applies every mutation together or none. It returns { ok: true, versionstamp } or { ok: false } when a check failed. It does not throw on a failed check; you must look at ok.

This is called optimistic concurrency: you do not lock anything. You read, compute, then commit with a check; if the world changed in between, you retry. It is fast when conflicts are rare.

The retry loop

Wrap read, compute and commit in a loop:

TypeScript
async function updateWithRetry<T>(
  kv: Deno.Kv,
  key: Deno.KvKey,
  update: (old: T | null) => T,
  attempts = 10,
): Promise<T> {
  for (let i = 0; i < attempts; i++) {
    const entry = await kv.get<T>(key);
    const next = update(entry.value);
    const res = await kv.atomic()
      .check(entry)           // an entry (key + versionstamp) works as a check
      .set(key, next)
      .commit();
    if (res.ok) return next;
  }
  throw new Error("Too many conflicts");
}

Passing the entry object you read straight to check is a handy shortcut, because it already has key and versionstamp.

Counters with sum

For a plain counter you do not need a read at all. sum adds to a 64-bit unsigned integer, inside the database, so concurrent increments never get lost:

TypeScript
await kv.atomic().sum(["stats", "visits"], 1n).commit();

const total = await kv.get<Deno.KvU64>(["stats", "visits"]);
console.log(total.value?.value);      // bigint, e.g. 41n

A sum value is a Deno.KvU64 (a wrapper around a bigint). Use bigint literals like 1n. min and max keep the smaller or larger value, which is handy for "best score" fields.

The stand-in below runs five concurrent increments two ways to show the difference between a naive read-modify-write and an atomic one.

Node.js Lost updates versus atomic updates (MiniKv stand-in)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

(The naive result varies from run to run, but it is almost never 5.)

Queues: do it later, reliably

A queue lets a request hand work to the background: send an email, resize an image, call a slow API, retry a webhook. You enqueue a message and a listener processes it, possibly on another machine.

TypeScript
const kv = await Deno.openKv();

// producer: runs inside a request handler
await kv.enqueue({ type: "welcome-email", userId: 7 });
await kv.enqueue({ type: "reminder", userId: 7 }, { delay: 24 * 60 * 60 * 1000 }); // in a day

// consumer: register once at startup
kv.listenQueue(async (message: { type: string; userId: number }) => {
  if (message.type === "welcome-email") {
    await sendEmail(message.userId);
  }
});

Delivery is at least once: if the handler throws, KV retries with backoff, so your handler must be idempotent (safe to run twice). For example, mark the job done with an atomic check so a duplicate delivery does nothing.

Options you will use:

Option Meaning
delay milliseconds to wait before the first delivery
backoffSchedule array of delays for retries, e.g. [1000, 5000, 10000]
keysIfUndelivered keys to write when every retry has failed (a dead-letter record)

You can also add enqueue to an atomic transaction (kv.atomic().set(...).enqueue(...)), which guarantees the job is queued if and only if the data was saved. That removes a whole class of bugs where the database has the order but no email job was created.

Watch: react to changes

kv.watch(keys) returns a ReadableStream. It emits the current values first, then a new array whenever any watched key changes:

TypeScript
const stream = kv.watch([["config", "theme"], ["config", "language"]]);

for await (const [theme, language] of stream) {
  console.log("theme:", theme.value, "language:", language.value);
}

Each emission is an array of entries in the same order as the keys you passed. If several writes happen quickly you may see only the latest state, so treat watch as "the value is now X", not as a log of every write. Typical uses: live dashboards, pushing updates to browsers over Server-Sent Events, and feature flags.

The stand-in below combines both features: a queue worker with retry, and a watcher fed by writes.

Node.js Queue with retry, and watch (MiniKv stand-in)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

Remote and Deploy notes

On Deno Deploy the same code uses a database replicated across regions, so reads can be local and fast while writes are coordinated. get and list accept { consistency: "eventual" } for faster but possibly slightly stale reads; the default is strong consistency. Use eventual reads for caches and public pages, never for data you are about to modify.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does atomic().commit() return when a check fails?

    A failed check does not throw. commit() resolves to { ok: false }, and your code decides whether to retry.
  2. 2. Which operation is best for a visitor counter that many requests increment at the same time?

    sum adds inside the database, so concurrent increments cannot overwrite each other.
  3. 3. Why must a queue handler be idempotent?

    KV retries failed deliveries, and a message can occasionally be delivered more than once, so repeating the work must be harmless.
  4. 4. What does a versionstamp of null in check() mean?

    check with versionstamp null succeeds only if the key is absent, which is how you create-if-not-exists.

Summary

  • kv.atomic() groups checks and mutations; commit() returns { ok } and applies everything or nothing.
  • Use optimistic concurrency: read, compute, commit with check(entry), retry when ok is false.
  • sum, min and max update 64-bit counters inside the database with no read.
  • enqueue and listenQueue run delayed, retried, at-least-once background jobs; attach enqueue to a transaction to keep data and jobs consistent.
  • watch streams the current and later values of keys.

Next: Data Formats and SQLite