Skip to content
Tutorials Center
My learning

Topics

Lesson 38 of 42 ยท Projects

Project: Cached API with Rate Limiting

Put Redis in front of a slow API: cache-aside with TTL, a lock that stops cache stampedes, a per-user token bucket, and hit-rate metrics, with live demos.

The most common reason teams add Redis to a system is to protect something slow or fragile behind it. A product page that needs three database queries gets slower every time traffic grows. An API that anyone can call in a loop eventually falls over because one client behaves badly. Redis helps with both, and with the same tool: very fast counters and keys that expire.

This project builds the data layer of a small product API with three protections. A cache answers repeated reads without touching the database. A lock makes sure that when a popular cache entry expires, only one request rebuilds it instead of hundreds at once (a cache stampede). A per-user token bucket limits how fast each client may call. On the side, a few counters record cache hits, misses and rejected requests, so you can see how well it all works.

You already know the building blocks: caching patterns, stampedes and invalidation and rate limiting. Here they work together. Real Redis cannot run on this page, so everything runs against a plain JavaScript emulator with the real semantics of GET, SET with NX, EX and PX, hashes and expiry. A fake clock and a small virtual-time scheduler let several "concurrent requests" sleep and wake up in a deterministic order, so the output is the same on every run. Business logic that would be a Lua script in real Redis is marked in the comments.

Three layers in front of the database

Limiter Token bucket per user.
Cache JSON with a 60 s TTL.
Lock One rebuild at a time.
Metrics Counters in a hash.
1 / 2

The request path

Every request passes the limiter first, because rejecting a client must be cheaper than serving it. Only a request that has a token goes on to the cache, and only a cache miss may touch the database.

flowchart TD
  A["Request"] --> B{"Token left?"}
  B -- "no" --> C["429 and Retry-After"]
  B -- "yes" --> D{"Cache hit?"}
  D -- "yes" --> E["Return cached product"]
  D -- "no" --> F{"SET lock NX PX"}
  F -- "got the lock" --> G["Load from database<br/>SET cache EX 60<br/>release lock"]
  F -- "someone else has it" --> H["Wait 50 ms<br/>look at the cache again"]
  H --> E

Limiter, then cache, then lock: each layer lets less traffic through to the next.

Cache-aside and the stampede lock

Cache-aside means the application owns the logic: read the key; on a miss, load from the database, store the result with a TTL and return it. Our key is cache:product:ID:v1. The v1 is a deliberate habit. When you change the shape of the cached JSON, bump it to v2 in the code, and the new release starts with a clean cache instead of reading entries it cannot parse. The 60-second EX bounds how stale anything can be even if you forget to invalidate.

Now picture a popular product whose entry expires. In the next 200 milliseconds, while the first request is still waiting for the database, a hundred more arrive, each sees a miss and each queries the database for the same row. That is a stampede, and it tends to happen exactly when the database is busiest. The cure is a short lock. SET lock:product:7 <token> NX PX 2000 succeeds for exactly one request. That request rebuilds the cache; the others wait 50 milliseconds, look at the cache again and find the fresh value. The lock has a PX 2000 expiry so a request that crashes mid-rebuild cannot block everybody forever.

Releasing the lock needs care. The code releases only if the lock still holds its own token, because after a long database call the lock may have expired and been taken by someone else; a plain DEL would delete their lock. Checking with GET and then deleting is two steps, so real code does it in one Lua script (see Lua scripting); the emulator's JavaScript is a stand-in.

The first TryIt fires six simultaneous requests for a cold product, once without the lock and once with it. The "database" takes 200 virtual milliseconds and counts its calls.

JS Six requests, one cold key: with and without the lock

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Without the lock, all six requests hit the database. With it, one request loads the product and the other five wait about 200 milliseconds in 50-millisecond steps and are served from the cache. The lock key is gone afterwards because the winner released it. The price is a little extra latency for the waiting requests, which is almost always better than a database falling over.

A lock is not the only defense. You can also refresh popular entries before they expire, serve slightly stale data while one request refreshes, or add random jitter to TTLs so that many keys do not expire in the same second. See cache invalidation and stampedes.

A token bucket per user

A token bucket gives each user a bucket that holds up to 5 tokens and refills at 1 token per second. Each request takes one token. A user who has been quiet can send a burst of 5 at once, but over time can only average 1 request per second. This is friendlier than a fixed window counter, which allows a double burst at the boundary between two windows.

The state of one bucket is two numbers: how many tokens it held and when we last looked. They live in a hash bucket:USER. On every request we compute the refill from the elapsed time (never above the capacity), subtract a token if one is available, and write the numbers back. Reading, computing and writing is a read-modify-write cycle, and two requests from the same user running at the same moment could both read the old count and both succeed. In real Redis the whole function therefore runs as a Lua script, which Redis executes atomically. Our takeToken function is the body of that script.

Look at the EXPIRE on the bucket: after capacity / refill seconds (5 here) a bucket would be full again anyway, so an expired bucket and a brand-new one behave identically. That means idle users cost no memory, and you never need a cleanup job.

When a request is rejected, answer with status 429 and a Retry-After header. The function computes it from the missing fraction of a token, so a polite client knows how long to wait.

Putting it together, with metrics

handleRequest runs the limiter, counts the outcome in metrics:api, and then calls getProduct, which counts hit or miss in metrics:cache. The cache hit rate is hit / (hit + miss), and it is the single best number for judging a cache. If it is low, the TTL may be too short, the keys may be too specific (for example, including a timestamp), or the cache may simply be too small.

The second TryIt plays out a short story with one user and another. ana sends a burst, is limited, waits for the bucket to refill and tries again; bo is unaffected by her; a minute later the cache entry expires and the next request reaches the database again.

JS Burst, 429, refill and hit-rate metrics

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Read the log with the diagram in mind. The first two requests miss the cache and go to the database (200 virtual milliseconds each), the next three are cache hits, and the sixth and seventh are rejected with a one-second Retry-After because the bucket is empty. bo's bucket is separate, so he succeeds and is served from the cache that ana's requests already filled. Two seconds later ana has earned two tokens, and exactly two requests pass before the third is refused. The final report shows the numbers an operator would watch: the hit rate, how many requests were allowed or limited, and how many times the database was really called.

Production notes

  • Decide how to fail. If Redis is down, should the limiter let everything through (fail open, protecting availability) or reject everything (fail closed, protecting the backend)? Most public APIs fail open for the cache and decide deliberately for the limiter. Set short client timeouts so a slow Redis does not slow every request.
  • Limit by the right identity. Per user id is fairer than per IP address, which punishes everyone behind one office network. Many APIs combine both.
  • Negative caching. Cache "not found" for a few seconds too, so requests for a missing product do not reach the database every time.
  • Metrics at scale. Counting every request into one hash key makes a hot key; for very high traffic, count in the application and flush totals periodically, or use INFO stats (keyspace_hits, keyspace_misses) for the server-wide hit rate.
  • Memory. Set maxmemory and an eviction policy for a cache-only instance (see memory limits and eviction).

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What does SET lock:product:7 token NX PX 2000 guarantee?

    NX makes the write succeed only for the first caller, and PX gives the lock a lifetime so a crashed holder cannot block others forever.
  2. 2. Why does the unlock code compare the stored token before deleting the lock?

    Each holder only releases its own lock. Doing the check and the delete atomically needs a Lua script in real Redis.
  3. 3. A user was idle for a minute, then sends 5 requests at once. What does a token bucket with capacity 5 do?

    A full bucket allows a burst up to its capacity. After that the user can only continue at the refill rate.
  4. 4. Why can the bucket hash safely have an EXPIRE of capacity divided by refill rate?

    A bucket that has had time to refill completely carries no information, so expiring it frees memory without changing behavior.
  5. 5. How is the cache hit rate calculated from the metrics hash?

    The hit rate is the share of lookups that were answered from the cache, hit / (hit + miss).

Summary

  • Put the limiter first, then the cache, then the lock: each layer protects the next one.
  • Cache-aside with a TTL and a versioned key (...:v1) is simple and safe to roll out.
  • SET key token NX PX ms is a cheap lock; release it only if the token still matches, in a Lua script in production.
  • A token bucket stored in a hash allows bursts and a steady rate, needs an atomic read-modify-write (Lua), and can simply expire when idle.
  • Count hits, misses and rejections with HINCRBY; the hit rate tells you whether the cache earns its memory.

Next project: Game Leaderboard Service.