Skip to content
Tutorials Center
My learning

Topics

Lesson 68 of 133 ยท Data and Storage

Caching in Memory

Build in-memory caches in Node.js - Map-based caches, TTL and LRU by hand, memoization, invalidation, stampede protection with shared promises, and WeakMap, WeakRef and FinalizationRegistry.

A cache is a place where you keep the result of expensive work so that you do not repeat it. The expensive work might be a database query, an HTTP request to another service, a heavy calculation or parsing a big file. The next time someone asks for the same thing, you answer from memory in microseconds instead of recomputing for milliseconds or seconds.

Caching is one of the most powerful speed-ups in software and also one of the most dangerous. A famous joke says there are only two hard problems in computer science: cache invalidation and naming things. The trouble is not storing values. It is deciding when a stored value has become wrong, and making sure the cache does not eat all your memory. In this lesson you will build the main kinds of in-memory cache by hand, see exactly how they behave, and learn the traps that production systems fall into.

All caches in this lesson live inside one Node.js process. If you run several processes or servers, each has its own copy, and you would reach for a shared store such as Redis (see Databases from Node.js and Scalability and Reliability Patterns). An in-process cache is still the fastest option there is: no network, no serialization.

The simplest cache: a Map

A Map already does the main job: look up by key, store by key. The pattern is "get, and if missing, compute and set". We count how often the slow function really runs, so the effect is visible without measuring any time.

Node.js A Map as a cache

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

Use cache.has(key) and not if (cache.get(key)). A legitimate cached value might be 0, '', false or null, and the truthiness check would treat it as a miss. This tiny cache has two weaknesses that every real cache must solve: it never expires, so data goes stale, and it never shrinks, so memory grows with every new key until the process dies. The next sections fix both.

Memoization

Memoization is caching applied to a pure function: same arguments, same result. You wrap the function once, and callers do not change. The only tricky part is the key. For one primitive argument, the argument itself works. For several arguments you need a string, often JSON.stringify(args). That fails for functions, undefined and objects with unstable property order, so use it only when your arguments are simple.

Node.js A generic memoize helper

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

Memoizing the recursive Fibonacci function turns exponential work into linear work: 51 runs (n = 0 to 50) for fib(50). This is the classic textbook case. In ordinary applications memoization shines for pure, deterministic and moderately expensive computations: formatting rules, parsed templates, compiled regular expressions, route tables.

Never memoize a function that depends on hidden state: the current time, a global variable, a database row or a random number. The cache would return an old answer forever. Also remember that a plain memoize has no size limit. Only use it for inputs with a small, bounded variety.

Expiration with a TTL

A TTL (time to live) puts a lifetime on each entry: after, say, 60 seconds the value counts as stale and is recomputed. It is the simplest invalidation strategy and it is right whenever a little staleness is acceptable (exchange rates, weather, a list of products).

To keep the demo deterministic we pass the clock as a function. This is also good design for testing, because you do not need real waiting. In production the default Date.now is used.

Node.js A TTL cache with an injectable clock

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

Expired entries are removed in two ways. Lazy removal happens when someone reads the key. That is cheap but never frees keys that nobody asks for again. Active removal (sweep) walks the map now and then, for instance from a setInterval. If you do that, call .unref() on the timer so it cannot keep the process alive by itself (see Timers).

Limiting size with LRU

A TTL does not bound memory: a million different keys within one TTL window are a million entries. A size limit does. When the cache is full, which entry should go? The popular answer is the least recently used (LRU): the one nobody touched for the longest time. Recently used data is likely to be used again soon, so this keeps the useful part.

A Map iterates in insertion order. That gives a very short LRU: on every read, delete and re-insert the key so it moves to the end ("most recent"); when the map is too big, delete the first key ("least recent").

Node.js An LRU cache in about 25 lines

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

Both operations are O(1) on average. For production, the lru-cache package (read-only mention here; the editor has no npm packages) adds a TTL, size-by-bytes limits and statistics. Combining a TTL with an LRU is the most common configuration: entries leave because they are old or because they do not fit.

Cache invalidation

You have three basic tools, and they are often combined.

  • Time-based (TTL). Simple, but data can be wrong for up to one TTL.
  • Explicit. When you change the source, delete the cached copy: cache.delete(userId) after updating a user. It is exact but only works if every write path remembers to do it, and only across processes if you have a way to tell the others.
  • Versioned keys. Put a version or a content hash in the key (user:42:v7, template:${hash}). A change creates a new key, and old keys simply age out. Nothing has to be deleted.

Also decide what to do about negative results. If a lookup finds nothing, should you cache "not found"? Often yes, for a short time, otherwise a missing key hits the database on every request. Store an explicit marker such as null, and use has to tell it from "never looked up".

Stampedes: many callers, one computation

Here is a subtle problem. Suppose a popular entry expires and 200 requests arrive in the same second. Each sees a miss and each starts the expensive query: the database gets 200 identical queries at the moment it is least able to cope. This is a cache stampede (also "thundering herd").

The remedy is to cache the promise, not the value. The first caller starts the work and stores the pending promise; everyone else gets the same promise and waits for the single result.

Node.js Stampede protection with a shared promise

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

The detail that matters is where set happens. The naive version stores the value only after await, so five callers all see an empty cache before any of them finishes. The shared version stores the promise synchronously, before anything can interleave. The .catch that deletes the entry is equally important: without it, one failed request would be cached forever and every later caller would get the same rejection.

Putting it together: an async cache with TTL and LRU

The pieces combine into one reusable helper, cached(key, loader). Failures are not cached. The clock is injected again so the run is deterministic.

Node.js A reusable async cache

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

That is a small but realistic cache. When you design your own, ask these questions: What is the key? How stale may the data be? What bounds the memory? What happens on failure? What happens when many callers arrive at once?

WeakMap: caches that clean up after themselves

Sometimes the key is an object, and you want the cached data to disappear when the object is no longer used anywhere else. A WeakMap holds its keys weakly: it does not keep them alive. When the key object is garbage-collected, the entry vanishes. This is ideal for attaching derived data to objects you do not own, such as a parsed form of a request or a computed layout. Keys must be objects (or non-registered symbols), and you cannot iterate or count a WeakMap.

Node.js Per-object results in a WeakMap

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

The copy configB counts as a different key because object identity is what matters. If you need to cache by content, build a string key (a hash or JSON.stringify) and use a Map with a size limit.

WeakRef and FinalizationRegistry

A WeakRef lets you hold a value that the garbage collector may free if nothing else uses it. You read it with ref.deref(), which returns the object or undefined if it has been collected. Combined with a Map, it makes a cache that keeps entries only while memory allows. A FinalizationRegistry runs a callback after an object was collected, so you can remove the dead map entry.

These tools exist, but use them sparingly. When garbage collection runs is decided by the engine, and the specification explicitly says code must not depend on it. Output and timing differ between runs, so we show a read-only example and a safe, deterministic one.

NODE
// Read-only sketch of a weak value cache (collection timing varies per run)
class WeakValueCache {
  #map = new Map();
  #registry = new FinalizationRegistry((key) => {
    const ref = this.#map.get(key);
    if (ref && ref.deref() === undefined) {
      this.#map.delete(key);          // clean up the dead entry
    }
  });

  set(key, object) {
    this.#map.set(key, new WeakRef(object));
    this.#registry.register(object, key);
  }

  get(key) {
    return this.#map.get(key)?.deref();
  }
}

The next example uses WeakRef but only prints things that cannot change between runs: while we still hold a normal reference, deref() always returns the object.

Node.js WeakRef while a strong reference exists

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

For most caches a Map plus a size limit is easier to reason about than weak references. Reach for WeakMap when the lifetime of the cached data should follow an object. Reach for WeakRef only when you can tolerate losing an entry at any moment, and never as a replacement for a size limit.

Memory: watch what you keep

A cache is deliberately a memory leak with a plan. Check that the plan holds. Bound the number of entries (LRU) and, for big values, the size. Do not cache whole request objects or database result sets when you only need two fields: the cached value keeps everything it references alive. Closures are another hidden source, since a function stored in a cache keeps its entire scope. Later lessons on V8 and memory show how to find leaks.

When to cache, and when not to

Cache when the work is expensive, the same answers are requested again and again, and a little staleness is acceptable. Do not cache when data must always be exact (a bank balance before a transfer), when keys are almost never repeated (the hit rate would be near zero and you only waste memory), or when the work is so cheap that a lookup costs as much as the computation. Measure the hit rate: hits / (hits + misses). A cache with a hit rate of 3 percent is decoration.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Why should a Map cache check cache.has(key) instead of if (cache.get(key))?

    A truthiness test treats legitimate values such as 0 or an empty string as missing. has tells 'present' from 'absent' exactly.
  2. 2. In the LRU cache built with a Map, why do you delete and re-insert a key on read?

    A Map iterates in insertion order, so the first key is the least recently used. Re-inserting on access keeps that order accurate.
  3. 3. What problem does caching the promise instead of the value solve?

    Storing the promise synchronously means later callers find it and wait for the same result instead of starting their own.
  4. 4. What should happen to a cached promise that rejects?

    Keeping a rejected promise would make every caller fail with the same error until the entry expired. Remove it unless you deliberately cache failures briefly.
  5. 5. Which statement about WeakMap is true?

    WeakMap keys must be objects and are held weakly. You cannot iterate or count the entries, because they may vanish at any time.

Summary

  • A cache trades memory for speed. Plan how entries expire (TTL, explicit delete, versioned keys) and how the size is bounded (LRU).
  • Map is the base: use has to detect hits, and remember it never shrinks by itself.
  • Memoization caches pure functions; build keys carefully and never memoize functions that depend on hidden state.
  • An LRU cache needs only a Map: delete and re-insert on use, and evict the first key when the cache is full.
  • Cache the promise, not the value, to avoid stampedes, and remove rejected promises.
  • Inject a clock to make TTL logic testable and deterministic.
  • WeakMap ties cached data to an object's lifetime; WeakRef and FinalizationRegistry are last-resort tools because collection timing is not predictable.
  • Track the hit rate. If it is low, remove the cache.

Next: Databases from Node.js