Skip to content
Tutorials Center
My learning

Topics

Lesson 32 of 37 ยท Patterns and Comparisons

Multi-Layer Caching and CDNs

Stack browser, CDN, in-process and Memcached caches, compute the effective latency and database load of each layer, and keep their TTLs and invalidation under control.

Memcached rarely works alone. A request for a web page may pass through a browser cache, a CDN edge, a reverse proxy, a per-process memory cache and Memcached before it ever reaches the database. Each layer is closer to the user, faster and smaller than the next. The art is to give each layer a clear job and to make sure they do not fight each other with conflicting lifetimes.

This lesson maps the layers, computes what each adds to latency and removes from the database load, and covers the practical problem: invalidating data that lives in several places. The examples are pure Python models, since real CDNs and Memcached servers are not available in the live editors.

The layers

flowchart LR
  B["Browser cache"] --> C["CDN / edge"]
  C --> P["Reverse proxy"]
  P --> L1["App memory (L1)"]
  L1 --> M["Memcached (L2)"]
  M --> D["Database"]

A request stops at the first layer that has a fresh answer. Only a miss at every layer reaches the database.

Layer Typical latency Shared by Good for
Browser 0 ms (no request) One user Static assets, immutable files
CDN edge 5 to 50 ms Everyone near that edge Anonymous pages, images, API responses safe to share
App memory (L1) microseconds One process Hot values, configuration, tiny TTLs
Memcached (L2) about 0.2 to 2 ms inside a data centre All app servers Objects, fragments, query results
Database 2 to 100+ ms Everyone Source of truth

The numbers vary widely by setup, but the order of magnitude is the point.

Effective latency and database load

If layer i has hit ratio h_i (counting only requests that reach it), the chance a request reaches the database is the product of the miss ratios. The model below shows how quickly a few modest layers add up to a huge reduction.

Python How layers multiply

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Notice that the second layer, even with a 90% hit ratio, takes care of most of the database load, and the L1 layer then trims a further slice and lowers the average latency. Each extra layer also adds complexity, so add one only when the numbers justify it.

HTTP caching headers: browser and CDN

The first two layers are controlled by response headers, not Memcached:

  • Cache-Control: public, max-age=300 allows browsers and shared caches to reuse the response for five minutes.
  • s-maxage sets a lifetime for shared caches (CDNs) separately from browsers.
  • private forbids shared caches: use it for per-user pages.
  • no-store forbids storing at all: use it for sensitive responses.
  • stale-while-revalidate=60 lets caches serve an old copy for up to a minute while fetching a fresh one in the background, the same idea as in Cache Stampede Protection.
  • ETag and Last-Modified allow cheap conditional requests: the cache asks "has it changed?" and gets a tiny 304 Not Modified if not.

Static files should use fingerprinted names (app.4f2a9c.js) with a very long max-age, so a change simply produces a new URL and nothing has to be invalidated.

A CDN that caches a page with a Set-Cookie header or personal data can show one user's page to another. Mark personalised responses private or no-store, and keep anonymous and logged-in variants separate.

The TTL budget

When layers are stacked, the worst-case staleness is roughly the sum of the TTLs. If the CDN holds a page for 60 seconds, Memcached for 60 seconds and the browser for 60 seconds, then a user may see data that is up to three minutes old after a change. Decide the freshness you promise, then divide it among the layers.

Python Worst-case staleness of stacked TTLs

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Invalidation across layers

Invalidation gets harder with each layer. Practical rules:

  1. Delete from the inside out when a record changes: database commit, then Memcached delete, then CDN purge (an API call or a surrogate-key purge) if you cache full pages there. An inside-out order avoids a CDN refilling from a stale L2.
  2. Prefer versioned URLs and versioned keys to purges (see Consistency Models and CAS): changing the name is instant and cannot race.
  3. Keep L1 TTLs tiny (a few seconds). There is no cheap way to invalidate a copy that lives in every process of every server.
  4. Surrogate keys / cache tags let you purge every CDN object that depends on a record ("all pages mentioning product 42") with one call, if your CDN supports them.
  5. Do not cache errors for long. A 500 page held by the CDN for ten minutes prolongs an outage. Use short TTLs for error responses.

Caching in the application framework

Most frameworks hide Memcached behind a cache facade (Cache::remember, cache.get_or_set, Django's cache, Rails' Rails.cache.fetch). Use the facade's remember helper (read, compute on a miss, store) for object caching, and let it add a TTL. Put expensive derived data (aggregates, rendered fragments) behind it; leave raw rows to the database's own buffer cache.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. A request stops at which layer?

    Layers are tried from the closest to the user inward, and the first hit answers.
  2. 2. What is the worst-case staleness when a browser, CDN and Memcached hold 30, 60 and 120 seconds?

    A change can sit in every layer for its full TTL one after another, so the lifetimes add up.
  3. 3. Why use fingerprinted file names such as app.4f2a9c.js with long max-age?

    Changing the name sidesteps invalidation entirely.
  4. 4. Which Cache-Control value should a personalised account page use?

    Personal pages must not be stored by shared caches such as a CDN.

Summary

  • Layers (browser, CDN, app memory, Memcached, database) each remove load from the next.
  • The chance of reaching the database is the product of the layers' miss ratios.
  • HTTP headers control the browser and CDN; stale-while-revalidate and ETag reduce rebuild cost.
  • Worst-case staleness is the sum of the layer TTLs; budget it.
  • Invalidate from the inside out, prefer versioned names, and keep L1 TTLs tiny.

Next: Serialization and Compression