Skip to content
Tutorials Center
My learning

Topics

Lesson 26 of 42 ยท Caching and Application Patterns

Redis in System Design, Distributed Cache and Session Store

Design with Redis: a distributed cache layer, multi-level caching with a near cache, a central session store, and back-of-the-envelope capacity and throughput estimates.

System design questions ask for the shape of a whole system: what runs where, what happens at ten times the traffic, what breaks first. Redis appears in almost every answer, because it is the standard shared fast layer between many application servers and a slower database. This lesson turns the individual patterns of this chapter into designs: a distributed cache, a multi-level cache with a local "near" cache, a central session store, and the arithmetic to size it all.

Everything here is runnable as JavaScript models, because the point is the reasoning, not a client library. The models use fake clocks and simple formulas, so they show trends rather than real benchmark numbers.

Why a shared cache layer?

A single web server can keep a cache in its own memory, and for a small site that is fine. With several servers it breaks down in three ways:

  • Each server warms its own copy, so you pay the database cost once per server and the hit rate falls as you add servers.
  • Invalidation is hard: after an update, one server clears its copy while the others keep serving the old value.
  • Memory is wasted: the same hot data sits in every process.

A distributed cache puts the data in one shared place that every server reaches over the network. Redis does that job: a read costs a fraction of a millisecond on a local network, ten to a hundred times cheaper than a typical database query.

flowchart LR
  U["Users"] --> LB["Load balancer"]
  LB --> A1["App server 1"]
  LB --> A2["App server 2"]
  LB --> A3["App server 3"]
  A1 --> R["Redis (shared cache)"]
  A2 --> R
  A3 --> R
  R -. "miss" .-> DB[("Database")]

Every app server asks the same Redis first, and only a miss reaches the database.

Designing the cache layer

When you put a cache into a design, answer these questions out loud:

  1. What is cached? Read-heavy, expensive-to-compute, slowly changing data: product pages, user profiles, query results, rendered fragments. Not data that must be exact and fresh (an account balance during a payment).
  2. Which pattern? Usually cache-aside: read the cache, on a miss load from the database and store with a TTL. Write-through or write-behind fit special cases.
  3. How does it expire? A TTL as a safety net plus explicit deletes on write. Add random jitter to TTLs so keys do not all expire together (see stampedes).
  4. What is the eviction policy? allkeys-lru or allkeys-lfu for a pure cache (see eviction and memory).
  5. What if Redis is down? The application falls back to the database, so the database must survive the extra load for a while, or you need rate limiting or load shedding in front of it. A cache that the database cannot live without is a single point of failure.
  6. How does it scale? One node first, then replicas for read throughput and Sentinel for failover, then Cluster when the data or write rate outgrows one machine. Spread hot keys by sharding them (product:42:shard:3) when a single key becomes the bottleneck.

Multi-level caching and the near cache

Redis is fast, but every call still crosses the network. For the hottest, smallest and least critical data you can add a second layer inside each application process, a near cache (or L1 cache) with a short TTL:

  • L1: a small in-process map or LRU, microseconds, per server, a few seconds to a minute of TTL.
  • L2: Redis, shared by all servers, about a millisecond, a minute to hours of TTL.
  • Source: the database.

A read checks L1, then L2, then the database, and fills the layers on the way back. The trade-off is staleness: two servers can disagree for as long as the L1 TTL. Keep L1 TTLs short, cache only data that tolerates that, and for stricter cases use Redis client-side caching (the server sends invalidation messages through CLIENT TRACKING, supported by most modern clients), which keeps L1 correct without waiting for a TTL.

The model below simulates 3 app servers reading a skewed set of 1,000 keys (a few keys are very popular), once with Redis only and once with an L1 near cache in front of it. It counts how many reads reached Redis and the database.

JS Redis only versus L1 + Redis + database

Press Run (or Ctrl + Enter) to see the console output here.

No output.

With these settings the near cache absorbs roughly a fifth of all reads locally (the exact count depends on the TTL and size you pick), and those are the popular keys, exactly where the load concentrates. The database numbers are the same in both runs only because this model never expires keys from Redis; the point to take away is how much traffic each layer absorbs, and that the L1 layer buys speed at the price of staleness.

A central session store

Web applications used to keep the session (who is logged in, what is in the cart) in the memory of the server that handled the login. Behind a load balancer that forces sticky sessions: the same user must always reach the same server, and if that server restarts or scales down, the user is logged out.

A central session store moves the session into Redis. The browser holds only an opaque session ID in a cookie; any server can load the session with one lookup:

Text
SET session:9f3a1c "{\"userId\":42,\"cart\":[7,9]}" EX 1800
GET session:9f3a1c
EXPIRE session:9f3a1c 1800     # slide the expiry on each request
DEL session:9f3a1c             # logout

Design points:

  • Stateless app servers. Servers can be added, removed or restarted at will, and the load balancer needs no stickiness.
  • Expiry does the cleanup. An idle timeout is just a TTL that you refresh. Add an absolute maximum lifetime as well.
  • Durability matters more than for a cache. Losing sessions logs everyone out. Use a separate Redis (or database number) with persistence and maxmemory-policy noeviction, plus a replica and failover, instead of the disposable page cache. See sessions and rate limiting for data layouts and security for access control.
  • Security. Make the ID long and random, set the cookie HttpOnly, Secure and SameSite, regenerate it at login, and keep secrets such as passwords out of the session.
  • Size. Keep sessions small (an ID and a few fields). Large sessions multiply memory and network cost.

Back-of-the-envelope capacity

System design answers are stronger with numbers. For Redis, estimate memory and operations per second.

Memory = items x (key size + value size + overhead). Redis adds roughly 50 to 100 bytes of overhead per key (the dictionary entry, the object header, expiry bookkeeping), more for small hashes and other structures. Multiply by a 1.3 to 1.5 safety factor for fragmentation, buffers and the fork copy-on-write cost during a background save, and leave headroom under maxmemory.

Throughput. One Redis instance typically handles tens of thousands to a hundred thousand or more simple commands per second per core, limited by the single command-execution thread, the network and command complexity. Estimate the request rate: daily active users x requests per user per day / 86,400 gives the average, and peaks are often 3 to 10 times higher. Each page view may need several Redis calls, so multiply.

JS Estimate memory and throughput for a session store and a cache

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Look at the second case: the data is about 16 GB (22 GB with headroom), which is large but fine for one box, while the peak operation rate is the number that argues for read replicas or a cluster. Always ask which resource, memory or operations, runs out first.

Putting the design together

A solid answer to "design a fast, scalable read path for a product site" mentions, in order: cache-aside in Redis with TTL, jitter and delete-on-write; a small L1 cache for the hottest keys; a separate persistent Redis for sessions; replicas and failover; monitoring of hit rate, memory, evictions and latency (see monitoring and troubleshooting); and a plan for Redis being down. Then give the numbers.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why does a distributed cache beat a per-server in-memory cache behind a load balancer?

    A shared cache avoids each server warming its own copy and serving different stale values. It is slower than local memory per call, but consistent and efficient.
  2. 2. What is the main cost of adding an L1 near cache in each app process?

    Each process has its own copy, so updates are not seen until the L1 entry expires (or client-side caching invalidates it).
  3. 3. Why should a session store use a different Redis setup from a page cache?

    A cache may evict entries by design, but evicting a session ends someone's login. Sessions deserve durability and no eviction.
  4. 4. What should you do about Redis overhead when estimating memory?

    Each key costs extra bytes beyond key and value, and background saves and fragmentation need headroom.

Summary

  • A shared Redis cache gives every app server one copy of hot data and one place to invalidate; the database only sees misses.
  • Decide what is cached, the pattern, the expiry (TTL plus jitter plus delete on write), the eviction policy and what happens when Redis is down.
  • A near (L1) cache cuts network calls for the hottest keys at the cost of bounded staleness; add client-side caching for stricter freshness.
  • A central session store makes app servers stateless; give it persistence, noeviction and failover, unlike a disposable cache.
  • Back your design with numbers: items x (key + value + overhead) x safety factor for memory, and peak operations per second for throughput.

Next, apply the patterns in a full build: Project: a URL shortener.