Skip to content
Tutorials Center
My learning

Topics

Lesson 24 of 42 ยท Caching and Application Patterns

Queues, Distributed Locks and Leaderboards

Build a reliable Redis queue with LMOVE, delayed jobs with sorted sets, safe SET NX PX locks with fencing tokens, and real-time leaderboards, with demos.

Beyond caching, Redis is often the glue between processes: a queue hands work from web requests to background workers, a lock makes sure only one process does a job at a time, and a leaderboard ranks things that change constantly. Each is a small recipe built from structures you already know: lists, sorted sets and strings with expiry.

The recipes look easy and hide sharp edges. A worker can crash after taking a job. A lock can expire while its owner is still working. A leaderboard can rank two players with the same score in a way that feels unfair. This lesson shows each recipe, the failure that breaks the naive version, and the fix.

Real Redis cannot run on this page, so redis-cli and Lua snippets are read-only. The runnable examples use a JavaScript stand-in that implements the command semantics involved (list moves, sorted set ranges, SET NX PX, a fake clock), so the failure scenarios replay the same way every time.

Three recipes

Queue A list moves jobs between steps.
Lock SET NX PX lets one client win.
Leaderboard A sorted set ranks by score.
1 / 2

A reliable queue with LMOVE

The simplest queue is a list: producers LPUSH queue job, workers RPOP queue. Jobs come out in the order they went in (first in, first out). Workers that would otherwise poll in a loop can block with BRPOP queue 5, waiting up to five seconds for a job.

The flaw is crash loss: RPOP removes the job from Redis. If the worker dies before finishing, the job is gone. The fix is to move the job instead of removing it. LMOVE queue processing RIGHT LEFT atomically pops the oldest job from queue and pushes it onto a second list, processing; its blocking twin is BLMOVE. The older RPOPLPUSH does the same thing. A worker then does the job and removes it from processing with LREM processing 1 job. If the worker crashes, the job is still sitting in processing, and a reaper moves stale entries back to the queue.

TEXT
127.0.0.1:6379> LPUSH queue "email:1" "email:2"
(integer) 2
127.0.0.1:6379> LMOVE queue processing:worker-a RIGHT LEFT
"email:1"
127.0.0.1:6379> LREM processing:worker-a 1 "email:1"
(integer) 1

Using one processing list per worker (processing:worker-a) tells the reaper which jobs belong to a dead worker. A real reaper pairs this with a heartbeat key with a short expiry that each worker refreshes, so "dead" is detectable. The result is at-least-once delivery: a job may run twice (the worker finished but crashed before LREM), so jobs must be idempotent (safe to repeat).

JS A queue that survives a worker crash

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Notice email:2 was never lost: it sat in processing:A until the reaper returned it. If you do not want to build this yourself, Redis Streams with consumer groups (see streams) give acknowledgements and pending lists as built-in features, and queue libraries such as Sidekiq, BullMQ and Celery's Redis broker package these ideas.

Delayed jobs with a sorted set

"Send this reminder in ten minutes" does not fit a list. A sorted set does: the member is the job, the score is the time it should run. ZADD delayed 1700000600 "reminder:5" schedules it. A scheduler loop asks for jobs whose score is at or before now, ZRANGE delayed -inf <now> BYSCORE LIMIT 0 10, and moves each due job to the ready queue. Because several scheduler processes may run, each must claim a job before moving it. ZREM returns 1 only for the caller that actually removed the member, so "the one that gets 1 owns it" is a cheap claim. (A Lua script doing read, remove and push in one step is the fully atomic version.)

JS Delayed jobs and the ZREM claim

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Every job ends up in the ready queue exactly once even though both pollers saw the same two due jobs.

Distributed locks

A lock lets many processes agree that only one of them runs a critical section: nightly report, cache rebuild, a payment capture. The basic form is one command:

TEXT
127.0.0.1:6379> SET lock:report 7f3a9c NX PX 30000
OK
127.0.0.1:6379> SET lock:report other NX PX 30000
(nil)

NX means "only if the key does not exist", so exactly one caller gets OK; PX 30000 gives the lock a 30 second lifetime so a crashed holder cannot block everyone forever. The value is a random token unique to this holder. Two rules follow.

  1. Release only your own lock. If you simply DEL lock:report, you might delete a lock that expired and was acquired by someone else. Check the token first, atomically, in a script:
LUA
if redis.call('GET', KEYS[1]) == ARGV[1] then
  return redis.call('DEL', KEYS[1])
else
  return 0
end
  1. Expect the lock to expire. If the work can outlast the TTL (a long garbage collection pause, a slow network call), the lock expires while you are still working, and a second process enters. Extend the TTL while working (again only if the token matches), and for resources that can check, use a fencing token: a number from INCR obtained with the lock. The protected resource rejects any write carrying a lower number than one it has already seen, so a stale holder is refused even if it wakes up late.
JS Unsafe release, safe release and a fencing token

Press Run (or Ctrl + Enter) to see the console output here.

No output.

In the unsafe run, A's late DEL removes B's lock and C walks in while B is working. With the token check, A's release does nothing and C is refused. In both runs the fencing token makes the storage reject A's late write.

Redlock and its caveats

A single Redis node as lock server is simple, but if it fails and a replica is promoted, the replica may not have received the lock yet (replication is asynchronous), so a second client can acquire it. Redlock tries to fix that by acquiring the lock on a majority of several independent Redis nodes within a short time. Its safety depends on assumptions about clocks and pauses that critics consider too strong. A practical summary: Redis locks are good for efficiency (avoid duplicate work, throttle a job) and risky as the only guard for correctness (money, inventory). For correctness, combine the lock with fencing tokens or use a transaction or constraint in the system of record, or a consensus-based store.

Leaderboards

Sorted sets are the natural leaderboard (the sorted sets lesson covers the commands). ZINCRBY board 150 player:7 adds points atomically, ZRANGE board 0 9 REV WITHSCORES gives the top ten, ZREVRANK gives a player's position, and a window of neighbours around that rank gives an around me view. All are O(log N) plus the size of the answer.

Ties need a decision. Players with equal scores are ordered by member name, and with REV that order is reversed, so the player whose name sorts later appears first regardless of who reached the score first. To rank "earliest to reach the score" higher, put a time component into the score: points * 10^10 + (10^10 - timestamp). Scores are doubles with 53 bits of exact integer precision, so this limits points to about 900,000; use a smaller time unit or fewer digits if you need more. Because the score now depends on the player's total, store the total separately (a hash) and update both in a script.

JS Top list, around me and fair tie-breaking

Press Run (or Ctrl + Enter) to see the console output here.

No output.

The naive board puts bo above ana, while the encoded score ranks ana first because she reached 500 earlier. The around-me window is just a ZREVRANGE from rank - 2 to rank + 2. Daily and weekly boards combine these ideas with ZUNIONSTORE and expiry, which the leaderboard project later in this track builds.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Why use LMOVE (or RPOPLPUSH) instead of RPOP for a work queue?

    RPOP deletes the job from Redis immediately. LMOVE atomically moves it to another list, so an unfinished job can be found and requeued.
  2. 2. Several scheduler processes poll a sorted set of delayed jobs. How do they avoid running a job twice?

    ZREM returns the number of members actually removed. Only one caller can remove a given member, so the return value works as a claim.
  3. 3. What should the value of a lock key created with SET NX PX be?

    A unique token lets the holder check, before releasing, that the lock is still its own and not one acquired by someone else after expiry.
  4. 4. What does a fencing token protect against?

    The resource remembers the highest token it has seen and refuses lower ones, so a holder that paused past its TTL cannot overwrite newer work.
  5. 5. Two players have the same score on a sorted-set leaderboard read with REV. Who appears first?

    Equal scores are ordered by member name, and REV reverses that order too. Encode a time component into the score to rank earlier achievers higher.

Summary

  • A list queue loses jobs on a crash if you pop with RPOP; LMOVE into a per-worker processing list plus a reaper gives at-least-once delivery, so jobs must be idempotent.
  • Delayed jobs are a sorted set scored by run time; pollers claim due jobs with ZREM.
  • Acquire locks with SET key token NX PX ttl, release only if the token matches (via a script), and expect expiry: extend while working and use fencing tokens where the resource can check them.
  • Redis locks suit efficiency; for strict correctness add fencing or rely on the system of record, and treat Redlock's assumptions carefully.
  • Leaderboards use ZINCRBY, ZRANGE ... REV, ZREVRANK and a rank window; encode time into the score to break ties fairly.

Next, structured documents and secondary indexes: Redis Stack: JSON, Search and Vectors.