Skip to content
Tutorials Center
My learning

Topics

Lesson 30 of 52 ยท Data Access

Caching: Memory, Distributed and Output

Speed up ASP.NET Core with IMemoryCache, expiration policies, stampede protection, IDistributedCache, HTTP caching headers with ETags and output caching.

The fastest database query is the one you never run. Caching means keeping the result of expensive work (a database query, a call to another service, a rendered page) so the next request can reuse it. A good cache turns a 200 ms request into a 2 ms one and protects your database from traffic spikes. A bad cache serves stale data forever, eats all your memory or collapses under load at exactly the wrong moment.

ASP.NET Core offers caching at several levels, and this lesson walks through them from the inside out: the in-process memory cache, the distributed cache abstraction for several servers, HTTP caching headers that let browsers and proxies help, and output caching that stores whole responses. Every example here runs live, because IMemoryCache, IDistributedCache and output caching all ship with ASP.NET Core.

flowchart LR
    B["Browser cache<br/>(Cache-Control, ETag)"] --> P["CDN / proxy"]
    P --> O["Output cache<br/>(whole responses)"]
    O --> M["Memory or distributed cache<br/>(your data)"]
    M --> D["Database / other service"]

A request can be answered at any layer; the closer to the user, the cheaper the answer.

IMemoryCache: a dictionary with expiry

IMemoryCache stores objects in the memory of the current process. Register it with builder.Services.AddMemoryCache() and inject it wherever you need it. It behaves like a thread-safe dictionary with rules for when entries disappear.

The main method is GetOrCreateAsync(key, factory): if the key is present you get the cached value, otherwise your factory runs, its result is stored and returned. Inside the factory you configure the entry:

  • AbsoluteExpirationRelativeToNow removes the entry after a fixed time, no matter how often it is read. Use it to bound staleness: "never older than 5 minutes".
  • SlidingExpiration removes the entry if it has not been read for that long, and every read renews it. Use it for "keep it while someone is using it". Combine it with an absolute expiration, or a popular entry can live forever.
  • Size and MemoryCacheOptions.SizeLimit let you cap the cache. When you set a SizeLimit, every entry must declare a Size, and the cache evicts entries (lowest Priority first) when the limit is reached. Without a limit a cache keyed by user input can grow until the process runs out of memory.
  • Priority, AddExpirationToken and RegisterPostEvictionCallback control eviction order, tie an entry to a change token (for example a file watcher) and react when an entry leaves.
C# IMemoryCache with absolute and sliding expiration

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with Console.ReadLine()), it gets the next line from this box.

The session entry survives three reads spread over 900 ms because each read slid its expiry forward, then disappears once nobody touches it for half a second. The profile entry is different: its absolute limit does not move, so it expired even though it was read.

Cache values that can be recomputed, never your only copy of data. Choose keys that include everything the value depends on ($"profile:{userId}:{culture}"), because the most common caching bug is one user receiving another user's cached data.

Cache stampede: when everyone misses at once

Suppose a popular entry expires and 500 requests arrive in the next 100 ms. Each finds the cache empty, so each runs the slow factory and hits your database. This is a cache stampede (or thundering herd), and it tends to strike exactly when load is highest. The surprise is that GetOrCreateAsync does not prevent it: it is not atomic with respect to the factory, so concurrent misses each run their own copy.

The fix is to make only one caller do the work and let the others wait for its result. A per-key SemaphoreSlim with a second check inside the lock is the classic pattern.

C# A stampede, and a lock that stops it

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with Console.ReadLine()), it gets the next line from this box.

All 20 callers are started before the first query finishes, so the naive version runs the factory 20 times and the guarded one exactly once. Two refinements matter in production: keep the lock per key (one global lock would serialize unrelated work), and decide what a failed load does so that waiters do not all retry in lockstep. The HybridCache library (Microsoft.Extensions.Caching.Hybrid, a NuGet package, so read-only here) packages this idea: its GetOrCreateAsync combines an in-memory first level, an optional distributed second level and built-in stampede protection.

IDistributedCache: sharing between servers

An in-process cache has two limits: each server has its own copy, and everything is lost on restart. When you run several instances behind a load balancer, you want one cache they all share. IDistributedCache is the small abstraction for that: a key to byte array store with string helpers, with real implementations for Redis, SQL Server and others (NuGet packages).

C#
// Production: one line chooses the backing store
builder.Services.AddStackExchangeRedisCache(options =>
    options.Configuration = builder.Configuration.GetConnectionString("Redis"));

ASP.NET Core ships AddDistributedMemoryCache(), an in-process implementation of the same interface. It is not distributed, but it is perfect for development and tests because your code does not change when you switch to Redis. Because values travel over the network, you must serialize them yourself, usually to JSON.

C# IDistributedCache with JSON values

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with Console.ReadLine()), it gets the next line from this box.

Notice that the distributed cache does not tell you whether an entry was a hit through a rich API: you ask, you get null or bytes. That simplicity is the point. Keep these habits: version your cache keys when the stored shape changes (product:v2:7), put a short expiry on everything, and treat the cache as optional, so that a Redis outage slows the app instead of breaking it.

HTTP caching: let the browser help

The cheapest request is one that never reaches your server. Response headers tell browsers and CDNs what they may reuse:

  • Cache-Control: public, max-age=60 lets any cache reuse the response for 60 seconds. private restricts it to the user's browser; no-store forbids storing; no-cache means "store it, but revalidate before use".
  • ETag is a fingerprint of the response. The client sends it back in If-None-Match; if it still matches, the server answers 304 Not Modified with no body, saving bandwidth even though the request itself is still made.
C# Cache-Control and ETag revalidation

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with Console.ReadLine()), it gets the next line from this box.

Never mark personalized responses public, and do not cache responses that set cookies or depend on the Authorization header unless you know exactly what varies. When a response varies by a request header (for example Accept-Language), add a Vary header, or caches will serve one user's variant to everybody.

Output caching: store whole responses on the server

HTTP caching depends on clients. Output caching (AddOutputCache, UseOutputCache) stores complete responses in your server and replays them to everyone, so even a client that sends no headers benefits. You choose what to cache per endpoint with CacheOutput, and policies control expiry, which query values or headers create separate entries (VaryBy...) and tags you can use to evict many entries at once.

By default only GET and HEAD requests that return 200 are cached, and responses to authenticated requests or that set cookies are skipped, which protects you from accidentally caching private data.

sequenceDiagram
    participant C as Client
    participant O as UseOutputCache
    participant H as Endpoint
    C->>O: GET /time
    O->>H: cache miss: run endpoint
    H-->>O: 200 + body
    O-->>C: response (stored)
    C->>O: GET /time
    O-->>C: stored response (endpoint not called)

On a hit, the endpoint never runs; the stored response is replayed.

C# Output caching with vary-by-query and tag eviction

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with Console.ReadLine()), it gets the next line from this box.

Output caching is best for responses that are expensive and identical for many users (a product list, a home page, a report) and for taming traffic spikes. Evict by tag when the underlying data changes, as the refresh endpoint does. For responses that differ per user, either leave them out or vary the entry by user.

Choosing and using caches wisely

  • Measure first. Cache what is slow and requested often. A cache you cannot show to be useful is just another bug surface.
  • Decide staleness explicitly. Every cache answers "how old may this data be?". Write that number down as an expiration.
  • Invalidate deliberately. Expire by time when slightly stale is fine, remove or evict by tag when you know the data changed.
  • Protect against stampedes for hot, slow keys, and add jitter (a small random offset) to expirations of many similar entries so they do not all expire together.
  • Bound the memory. Use SizeLimit, or expirations, for any cache keyed by user-controlled input.
  • Cache copies, not shared mutable objects. If you cache a list and one request mutates it, every request sees the change.
  • Never cache secrets or per-user data under a shared key.

For API responses, the same ideas return in the Performance Tuning lesson and the caching parts of the projects at the end of this tutorial.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What is the difference between AbsoluteExpirationRelativeToNow and SlidingExpiration?

    Absolute expiration bounds the maximum age of an entry. Sliding expiration is renewed by every read, so an entry used often can live a long time unless you also set an absolute limit.
  2. 2. Why can GetOrCreateAsync still cause a cache stampede?

    Several callers that miss at the same time each execute their own factory. A per-key lock with a second check (or HybridCache) makes only one of them do the work.
  3. 3. When would you pick IDistributedCache over IMemoryCache?

    A distributed cache is an external store shared by all instances. It is slower than local memory because of serialization and the network, but consistent across servers.
  4. 4. A client sends If-None-Match with the ETag it received, and the resource has not changed. What does the server return?

    A matching ETag means the client's copy is still valid, so the server replies 304 and the client reuses what it already has.
  5. 5. What does output caching store, and who benefits from it?

    UseOutputCache keeps whole responses in the server's cache and replays them, so even clients that send no caching headers benefit. By default only GET and HEAD 200 responses without cookies or authorization are cached.

Summary

  • IMemoryCache is a thread-safe in-process dictionary with absolute and sliding expiration, size limits and priorities.
  • GetOrCreateAsync is convenient but does not stop a cache stampede; guard hot keys with a per-key lock and a second check, or use HybridCache.
  • IDistributedCache is a byte-array cache shared by several servers (Redis, SQL Server); you serialize values yourself, and AddDistributedMemoryCache is a development stand-in.
  • HTTP caching uses Cache-Control, ETag and 304 Not Modified to let browsers and CDNs reuse responses; add Vary when responses depend on request headers.
  • Output caching stores whole GET responses on the server, can vary by query or header and is evicted by tag.
  • Decide staleness, bound memory, include everything the value depends on in the key, and never share per-user data under a shared key.

Next: Authentication Concepts