Skip to content
Tutorials Center
My learning

Topics

Lesson 35 of 37 ยท Projects and Practice

Project: A Cache Cluster Simulator

Build a small Memcached-style cluster in Python: an LRU server with TTLs and stats, a consistent hash ring client, a traffic generator and a node failure experiment.

In this project you will put the whole chapter together by building a miniature Memcached cluster that runs entirely inside the live editor. It has three parts: a server with byte-based capacity, LRU eviction, TTLs and stats; a client with a consistent hash ring and virtual nodes; and a traffic simulator that replays skewed requests, takes a node down and reports the hit ratio and database load.

It is a teaching model. It uses a virtual clock rather than real time, a dictionary in place of sockets and a simplified single LRU instead of slab classes, so numbers will not match a real server. What it does reproduce faithfully are the behaviours you studied: placement, eviction, expiry, misses and the effects of failure.

Step 1: the server

A server stores items with an expiry time and a size, and evicts from the least recently used end when it would exceed its byte limit. Expired items are dropped lazily when they are read, like Memcached does for items it finds past their TTL.

Python Step 1: an LRU server with TTL and stats

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The oldest keys are gone, the newest remain, and the expired key counts as a miss. This is a small working model of what a real server does for every command.

Step 2: the cluster client

The client builds a hash ring from the servers, with virtual nodes, and routes every operation. It also supports removing a node (eject) and re-adding it, as in Node Failure and Failover.

Step 3: simulate a day of traffic

The full program below includes the server from step 1, a ring-based Cluster, a database stand-in that counts reads, and a cache-aside read() function. Requests follow a Zipf distribution. Halfway through, one node is taken down (and ejected from the ring); later it comes back empty.

Python Steps 2 and 3: cluster, traffic and a node failure

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

When you run it, compare the phases. The warm-up phase has a lower hit ratio and many database reads while the cache fills. The healthy phase settles at a high hit ratio with far fewer reads. When one of four nodes goes down, its keys are re-homed on the three survivors, which must fetch them from the database, so database reads rise by roughly half even though only a quarter of the keys moved. When the node comes back empty, the keys that return to it miss again, which produces a second bump. The table at the end shows that cache3 holds fewer items because it was away for a while. A rejoining node should be warmed or ramped (see Node Failure and Failover).

Step 4: extend it

Treat the simulator as a lab. Each of these extensions teaches one idea from this track:

  1. Change the capacity to 100,000 and 40,000 bytes (a cache smaller than the working set evicts constantly) and compare hit ratios (see Eviction Policies and Capacity Planning).
  2. Add jitter to the TTL (ttl * random.uniform(0.9, 1.1)) and compare the database read spikes after warm-up.
  3. Replace the LRU in Server with FIFO (drop move_to_end) and compare hit ratios.
  4. Add a gutter server that takes keys of the ejected node with a TTL of 10 seconds.
  5. Weight the servers: give cache4 twice the vnodes and double the capacity, and check the per-server item counts (Virtual Nodes and Load Balance).
  6. Add a hot key: make 5% of the requests use one key and measure the per-server hits to see the imbalance, then replicate it (Hot Keys and Replication).
flowchart LR
  T["Traffic generator (Zipf)"] --> R["read(): cache-aside"]
  R --> C["Cluster (hash ring)"]
  C --> S1["Server 1"]
  C --> S2["Server 2"]
  C --> S3["Server 3"]
  R -->|"miss"| DB["Database stand-in"]

The simulator's structure: a cache-aside read path over a hash ring of LRU servers.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why does the simulator count a read of an expired key as a miss?

    Expired items are not returned; they count as misses and are reclaimed.
  2. 2. What happens when the failed node rejoins the ring empty?

    The ring sends keys back to the cold node, so they miss until repopulated.
  3. 3. In the simulator, what is the effect of a smaller byte capacity per server?

    Less space means more items evicted before they are reused.
  4. 4. Which pattern does the read() function implement?

    The application reads the cache, falls back to the database and stores the result.

Summary

  • You built a server (LRU, TTL, stats), a ring-based client and a cache-aside read path.
  • Traffic skew, capacity and failure events all show up in hit ratio and database reads.
  • A node that rejoins empty causes a second dip; warm or ramp it.
  • The simulator is a lab for trying TTL jitter, gutter pools, weights and hot-key fixes.

Next: Project: A Cache Library with Stampede Protection