Skip to content
Tutorials Center
My learning

Topics

Lesson 133 of 182 ยท Dependency Injection and Hosting

Caching and HttpClientFactory

Learn in-memory caching with IMemoryCache - expiration, GetOrCreate, size limits, cache stampede - and typed HttpClients with IHttpClientFactory, handlers and resilience in .NET.

Two things make server applications slow and fragile: doing the same expensive work again and again, and talking to other services over the network. This lesson tackles both with the tools .NET gives you.

Caching means keeping the result of expensive work (a database query, a computed report, a downloaded file) in memory, so the next request can reuse it. IMemoryCache is the built-in in-process cache. It is simple to use and easy to use wrongly, so we will look at expiration, size limits and the notorious cache stampede.

IHttpClientFactory manages HttpClient instances for you: it pools the underlying connections, rotates them so DNS changes are noticed, and lets you configure clients once (base address, headers, handlers) and inject them by type or name. Together with a resilience pipeline for retries and timeouts, it is the standard way to call other HTTP services.

Both live in the ASP.NET Core shared framework that our runner includes (Microsoft.Extensions.Caching.Memory, Microsoft.Extensions.Http). In a console project of your own you add those two NuGet packages. The basics of HttpClient itself are in HttpClient.

IMemoryCache basics

IMemoryCache is a thread-safe dictionary-like store inside your process. You register it with services.AddMemoryCache() (ASP.NET Core does this for you) and ask for IMemoryCache in a constructor. The core operations are:

Method What it does
Set(key, value, options) Stores a value; returns the value.
TryGetValue(key, out value) Reads a value if present and not expired.
Remove(key) Deletes an entry.
GetOrCreate(key, factory) / GetOrCreateAsync Returns the cached value, or runs the factory, stores and returns its result.

Keys and values are object; the extension methods give you typed access.

C# Set, TryGetValue and GetOrCreate

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with Console.ReadLine()), it gets the next line from this box.

GetOrCreate is the method you will use most: it expresses "get it, or compute it once" in a single line. Outside dependency injection you can create a MemoryCache yourself, as above. Always dispose it (or let the container do so); it owns an internal timer for expiration scanning.

Expiration: how long is a value valid?

Cached data goes stale. You decide how long to trust it with entry options:

  • Absolute expiration: the entry dies at a fixed time, no matter how often it is read (AbsoluteExpirationRelativeToNow = TimeSpan.FromMinutes(5)).
  • Sliding expiration: the entry dies if it was not read for a period; every read pushes the deadline forward (SlidingExpiration).
  • Both: it dies at whichever comes first. This is the safe combination. A sliding expiration alone can keep a popular entry alive forever, serving data that is arbitrarily old.
C# Absolute and sliding expiration

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with Console.ReadLine()), it gets the next line from this box.

The absolute entry disappears after its 300 ms no matter how often we read it. The sliding entry survives as long as we keep reading it more often than every 300 ms, and dies only after a pause.

Expiry is checked lazily when you read, and by a background scan that runs about once a minute, so expired entries may linger in memory briefly but are never returned.

Eviction callbacks and removal reasons

Each entry can register a callback that runs when it leaves the cache. The EvictionReason tells you why: Expired, Removed, Replaced, Capacity, TokenExpired. Use it for logging or releasing resources.

C# Why did the entry leave?

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with Console.ReadLine()), it gets the next line from this box.

Callbacks run asynchronously on a thread pool thread, which is why the order of lines inside a busy program is not guaranteed. In this small example each action is separated by enough time that the output is stable.

Size limits: a cache must not eat all memory

By default IMemoryCache has no limit. Cache something with an unbounded set of keys (one per user, per URL, per search text) and memory grows until the server dies. Set a limit and give every entry a size:

C#
services.AddMemoryCache(options => options.SizeLimit = 1000);

The unit is up to you (items, kilobytes, ...). Once a limit is set, every entry must specify Size, otherwise Set throws an InvalidOperationException. When adding an entry would exceed the limit, the cache evicts others (least recently used, with priority taken into account) and may even refuse the new entry.

C# A cache with a size limit

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with Console.ReadLine()), it gets the next line from this box.

You end up with at most three entries: the limit is enforced even though five were added. The exception message tells you exactly what is wrong. Which items survive is an implementation detail (roughly least-recently-used first), so never depend on it.

Other useful options: Priority (CacheItemPriority.NeverRemove for critical items), and change tokens to evict entries when something else changes, for example a CancellationTokenSource whose Cancel() invalidates a group of related entries:

C# Evict a group of entries with a token

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with Console.ReadLine()), it gets the next line from this box.

One Cancel() removed all three entries (eviction by token happens on a thread-pool thread, so we waited a moment before counting). This is the standard way to clear a family of related keys without enumerating them (IMemoryCache deliberately has no "list all keys" method).

GetOrCreateAsync and the cache stampede

For expensive asynchronous work, GetOrCreateAsync does the same job. But here lies a famous trap. Imagine a popular page whose data takes two seconds to load, and the cache entry has just expired. A hundred requests arrive in that moment. Each finds the cache empty, so each starts the expensive load. Your database receives a hundred identical queries exactly when it is already struggling. That is a cache stampede (also called thundering herd or dog-piling).

GetOrCreateAsync does not prevent this: it is not atomic with respect to the factory. Let's prove it:

C# The stampede: the factory runs many times

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with Console.ReadLine()), it gets the next line from this box.

Ten callers, ten expensive runs. The result is correct, but the work is wasted. The fix is to make concurrent callers for the same key share one in-flight load. The simplest version caches a Lazy<Task<T>> (or the Task<T> itself), so the first caller starts the work and all others await the same task:

C# Fix: share one in-flight task

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with Console.ReadLine()), it gets the next line from this box.

The lock is held only while starting the task, not while awaiting it, so it is fast. Every caller now receives the same Task<string> and the factory runs once. Two refinements for production code:

  • A failed task is cached too. If factory() throws, every caller keeps getting that exception until the entry expires. Remove the entry when the task faults (a continuation that calls cache.Remove(key) on failure).
  • Cancellation. If the first caller cancels and the shared task honors its token, everyone else is cancelled too. Use a token owned by the cache, not by one request.

For anything more demanding, .NET 9 introduced HybridCache (Microsoft.Extensions.Caching.Hybrid, a NuGet package). Its GetOrCreateAsync has built-in stampede protection and combines a fast in-memory first level with an optional distributed second level such as Redis. It is the recommended choice for new ASP.NET Core applications, while IMemoryCache remains perfect for simple, single-server cases.

C#
// Read-only: needs the Microsoft.Extensions.Caching.Hybrid NuGet package.
builder.Services.AddHybridCache();

// later, in a service
string report = await hybridCache.GetOrCreateAsync(
    "report",
    async cancellationToken => await LoadReportAsync(cancellationToken));

Pitfalls with caching

  • Mutable cached objects. The cache stores a reference. If one request modifies the cached List<T>, all others see the change (and may race). Cache immutable data, or copy it.
  • Caching per user without a limit. Unbounded keys mean unbounded memory. Use SizeLimit.
  • Caching null or errors by accident. Decide deliberately whether "not found" should be cached (often yes, briefly, so missing data doesn't hammer the database).
  • Multiple servers. IMemoryCache lives in one process. With three web servers you have three separate caches that can disagree. Use a distributed cache (IDistributedCache, Redis) when that matters.
  • The golden rule of caches: "There are only two hard things in computer science: cache invalidation and naming things." Always ask how a value becomes wrong, and make sure an expiry or a token handles it.

IHttpClientFactory: why it exists

Creating new HttpClient() for every call looks harmless but is a classic bug. Each instance owns a connection pool; disposing it leaves sockets in the TIME_WAIT state, and under load you run out of ports (SocketException). The opposite fix, one static HttpClient forever, keeps connections alive for too long, so it never notices a DNS change.

IHttpClientFactory solves both. It keeps a pool of HttpMessageHandler objects (the part that owns connections), recycles each handler about every two minutes, and hands out cheap, short-lived HttpClient wrappers. You register clients in one place and receive configured instances by injection.

The method comes from Microsoft.Extensions.DependencyInjection.AddHttpClient. To make this runnable, we start a tiny Kestrel server on loopback (more on that in ASP.NET Core Overview) and call it.

Named and typed clients

A named client is fetched by a string from the factory. A typed client is a class that receives an HttpClient in its constructor, configured for its purpose. The typed form is usually nicer: the class wraps the HTTP details and exposes methods that talk in your domain's language.

C# A typed client against a local server

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with Console.ReadLine()), it gets the next line from this box.

The typed client's settings (base address, header, timeout) apply to every call it makes; the plain named client has only the base address, so the X-App header shows up empty. AddHttpClient<WeatherClient> registers WeatherClient as a transient service, which is fine because the HttpClient inside it is cheap while the heavy handler is pooled.

Two rules:

  • Don't dispose factory clients with the idea of freeing sockets; disposal is harmless but unnecessary. Handler lifetime is managed by the factory.
  • Don't store a typed client in a singleton. The injected HttpClient would live as long as the singleton, and the handler could never be recycled. Inject IHttpClientFactory into singletons and create clients as needed.

Handlers: cross-cutting behavior

A DelegatingHandler wraps every request and response, like middleware for outgoing calls. Typical uses: add an authentication token, log timings, retry, propagate a correlation id. You add handlers to the client's pipeline, outermost first.

C# A logging handler in the pipeline

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with Console.ReadLine()), it gets the next line from this box.

The handler sees the request on the way out, calls base.SendAsync to pass it down the chain (the last handler is the real network handler), and sees the response on the way back. Forgetting to call base.SendAsync would break the chain, since nothing would ever reach the server.

Factory-created clients also log each request and response through ILogger by default (category System.Net.Http.HttpClient.<name>.LogicalHandler), which is one more reason to know Logging.

Resilience: retries, timeouts and circuit breakers

Networks fail. Servers return 503, connections reset, responses take forever. A robust client needs a resilience strategy:

  • Timeout per attempt and for the whole operation, so a hung server cannot hang you.
  • Retry with exponential backoff and jitter for transient failures (timeouts, 5xx, 408, 429), never for errors that will repeat (400, 404).
  • Circuit breaker: after many failures in a row, stop calling the sick service for a while, so you do not pile onto it and you fail fast.
  • Rate limiter / bulkhead: cap concurrent calls.

The old standard library for this was Polly (also a NuGet package). Since .NET 8 there is the official integration Microsoft.Extensions.Http.Resilience, built on Polly v8. The simplest setup is one call that adds a sensible pipeline (rate limiter, total timeout, retry, circuit breaker, attempt timeout):

C#
// Read-only: needs the Microsoft.Extensions.Http.Resilience NuGet package,
// which our in-browser runner does not include.
builder.Services.AddHttpClient<WeatherClient>(c => c.BaseAddress = new Uri("https://api.example.com"))
       .AddStandardResilienceHandler();

// Or tune it:
builder.Services.AddHttpClient("payments")
       .AddStandardResilienceHandler(options =>
       {
           options.Retry.MaxRetryAttempts = 4;
           options.Retry.BackoffType = DelayBackoffType.Exponential;
           options.AttemptTimeout.Timeout = TimeSpan.FromSeconds(5);
           options.TotalRequestTimeout.Timeout = TimeSpan.FromSeconds(30);
       });

Since the package is not available here, you can still understand (and test) the idea with a hand-written retry handler. This one retries when the server answers 503, with growing delays, and it uses a loopback server that fails the first two times:

C# A hand-written retry handler

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with Console.ReadLine()), it gets the next line from this box.

Real libraries add jitter (random variation, so a hundred clients don't retry in lockstep), honor Retry-After headers, and only retry requests that are safe to repeat. A GET is naturally safe; a POST that creates an order is not unless the server supports an idempotency key. That is why you should prefer the maintained library over your own loop in production.

When to use what

  • IMemoryCache: single-server, simple data, reads far more frequent than writes. Add a SizeLimit and expirations.
  • HybridCache or a distributed cache: several servers, stampede protection needed, data larger than you want per process.
  • Typed clients: a class per external service, with methods named in your domain.
  • Named clients: quick configuration for small integrations.
  • Resilience handler: always, for any call across a network that you do not control.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What does sliding expiration alone risk?

    Each read pushes the deadline forward, so a popular entry never expires. Combine it with an absolute expiration to cap the maximum age.
  2. 2. You set SizeLimit on the cache and call cache.Set("k", value) without options. What happens?

    When SizeLimit is configured, every entry must declare a Size. Otherwise Set throws InvalidOperationException.
  3. 3. Ten requests call GetOrCreateAsync for the same missing key at the same moment. How often does the factory typically run?

    IMemoryCache is thread safe for its own data, but it does not serialise the factory. Concurrent misses can all run the expensive work: a cache stampede. Share one in-flight task or use HybridCache.
  4. 4. Why prefer IHttpClientFactory over creating a new HttpClient for every call?

    A new HttpClient per call creates a new connection pool each time and can exhaust ports. A single static one never refreshes DNS. The factory pools handlers and recycles them periodically.
  5. 5. In a DelegatingHandler, what happens if SendAsync never calls base.SendAsync?

    base.SendAsync passes the request to the inner handler, ultimately the network handler. Without that call you must return your own HttpResponseMessage.

Summary

  • IMemoryCache stores objects in the process; use GetOrCreate(Async) for "get or compute once".
  • Choose expiration deliberately: absolute caps the age, sliding keeps hot data, and both together are the safe default. Change tokens evict groups of entries.
  • Set a SizeLimit (and entry sizes) for caches whose key space is unbounded.
  • GetOrCreateAsync can run the factory several times for concurrent misses (cache stampede); share one in-flight task, or use HybridCache.
  • IHttpClientFactory pools and recycles handlers; use named or typed clients, and don't keep typed clients in singletons.
  • DelegatingHandlers add cross-cutting behavior; for real resilience use AddStandardResilienceHandler (Polly-based), with retries only for safe, transient failures.

Next: ASP.NET Core Overview