Skip to content
Tutorials Center
My learning

Topics

Lesson 45 of 77 ยท Middleware, Signals and Internals

Caching

Speed up Django with caching: configure backends, use get, set and get_or_set, cache whole views and template fragments, and invalidate stale data safely.

A cache is a fast place to keep the result of slow work, so the next request can reuse it instead of repeating the work. Rendering a page that runs ten database queries, calling a slow external service, or computing a report are all good candidates. A cache hit can turn hundreds of milliseconds into a fraction of a millisecond.

Django has a complete caching framework: a small API for storing values, ready-made backends for different storage systems, and higher-level tools that cache whole views or pieces of templates. This lesson walks through the layers from the bottom up. The live examples use the in-memory locmem backend, which behaves the same way as the others from your code's point of view.

There is a famous joke that the two hard problems in computing are naming things and cache invalidation. The last section of this lesson is about the second one.

Choosing a backend

You configure caches in the CACHES setting. The backend decides where values are stored:

Backend Stored in Good for
django.core.cache.backends.locmem.LocMemCache Process memory Development, tests, tiny sites
django.core.cache.backends.redis.RedisCache A Redis server Production, shared by all processes
django.core.cache.backends.memcached.PyLibMCCache Memcached Production, simple and fast
django.core.cache.backends.db.DatabaseCache A database table When you have no other cache server
django.core.cache.backends.filebased.FileBasedCache Files on disk Simple single-server setups
django.core.cache.backends.dummy.DummyCache Nowhere Turning caching off while keeping the code

The most important difference is sharing. locmem is private to each process, and a production server usually runs several worker processes, so each would have its own copy of the cache and invalidating one would not clear the others. For production you want Redis or Memcached, which every process connects to. A typical Redis setup (read-only here, because the browser has no Redis server):

Python
CACHES = {
    "default": {
        "BACKEND": "django.core.cache.backends.redis.RedisCache",
        "LOCATION": "redis://127.0.0.1:6379/1",
        "TIMEOUT": 300,            # default lifetime in seconds
        "KEY_PREFIX": "mysite",    # avoid clashes when sharing a server
    }
}

Because your code only talks to django.core.cache.cache, switching from locmem in development to Redis in production is a settings change.

The low-level cache API

The cache behaves like a dictionary with expiry times. The core methods are set, get, add, delete, get_or_set, incr, get_many and set_many. Every value has a timeout in seconds; after that the key disappears, and get() returns the default.

To show expiry without waiting, the example below replaces the clock with a fake one that we move by hand.

Python The low-level cache API

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Notice get_or_set: it returns the cached value if present, and otherwise calls your function, stores the result and returns it. It is the shortest way to write the common "check, compute, store" pattern. The report key had a 30-second life, so it vanished after 45 seconds, while greeting (60 seconds) survived until 65.

A few rules about the API:

  • Values must be picklable (strings, numbers, lists, dicts, most model instances). The cache stores a copy, so changing the original afterwards does not change the cached value.
  • get() returns None for a missing key, which is ambiguous if you cache None itself. Pass a unique sentinel as the default, or use get_or_set.
  • incr raises ValueError if the key does not exist, so set it first or use add to create it.
  • A timeout=None means "never expire", and timeout=0 means "do not store".
  • Keys are strings; keep them short, and build them in one helper function so every caller agrees on the format.

Caching a whole view

If an entire page is the same for everyone for a few minutes, decorate the view with cache_page. Django stores the full response, keyed by the URL and by any headers listed in Vary, and serves later requests without running the view at all.

Caching a piece of a template

Often only part of a page is expensive, such as a sidebar of popular articles. The {% cache %} template tag stores the rendered HTML of just that block. It takes a timeout in seconds, a fragment name, and optional extra values that become part of the key (so you can have one copy per language, or per user).

The next example shows both. A counter records how many times each expensive function really ran.

Python cache_page and fragment caching

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The report view ran once even though it was requested three times; the cached response was served for the rest. A different query string (?page=2) is a different cache entry, so it ran again. The fragment demo shows the important property of {% cache %}: the block is frozen, but everything outside it, like the title, stays live. Adding lang after the fragment name made French and English fragments separate.

Never use whole-view caching for pages that differ per user, such as a page showing the logged-in user's name or a form with a CSRF token. Django adds Vary: Cookie for pages that touch the session, which keeps entries apart, but it is easy to leak private data through a shared cache. When in doubt, cache a fragment or a data value instead of the whole page.

You can also apply caching to every page with the UpdateCacheMiddleware and FetchFromCacheMiddleware pair (the "per-site cache"), but per-view and fragment caching give you much more control.

Cache invalidation

Caching is easy; keeping the cache correct is the hard part. There are three basic strategies:

  1. Expiry only. Set a short timeout and accept slightly stale data. The simplest option, and often good enough for a "popular posts" box.
  2. Delete on change. When the underlying data changes, delete the cached key. Fresh data, but you must find every place that changes the data.
  3. Versioned keys. Include a version number in the key and bump it when the data changes; old entries just stop being read and expire on their own.

The final example combines caching of a database query with delete-on-change using a signal (see Signals), and counts the real queries to prove the cache works.

Python Caching a query and invalidating it

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The first call hits the database; the second costs zero queries. Saving or deleting a Product fires the signal, which removes the key, and the next call rebuilds it. This is simple and correct for a single list. For many related keys, versioned keys scale better than hunting down individual names.

Finally, remember cache stampedes: when a popular key expires, many requests may all miss at once and all recompute the value. For very hot keys, add a little random jitter to the timeout, or refresh the value from a background job before it expires.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Why is LocMemCache a poor choice for production with several worker processes?

    locmem lives in each process's memory. A shared backend like Redis or Memcached gives every worker the same cache.
  2. 2. What does cache.get_or_set('key', func, 60) do?

    It is the compact form of check, compute, store. The function runs only on a miss.
  3. 3. In '{% cache 300 sidebar lang %}', what is lang?

    Arguments after the fragment name vary the key, so each distinct value gets its own cached copy.
  4. 4. A product list is cached for 5 minutes, but admins want changes to show immediately. Which fix is most direct?

    Delete-on-change keeps the data fresh: the next read finds no key and rebuilds it from the database.
  5. 5. Why be careful with cache_page on pages that show the logged-in user's name?

    Whole-page caching stores the finished response. If the page contains per-user content, other visitors may receive it unless the cache is varied correctly.

Summary

  • Configure caches in CACHES; use locmem for development and Redis or Memcached when several processes must share one cache.
  • The API: set, get, add, get_or_set, incr, get_many, set_many, delete, clear, each key with a timeout.
  • cache_page stores whole responses by URL; {% cache %} stores rendered template fragments with optional key variations.
  • Invalidate by short expiry, by deleting keys when data changes, or by versioned keys.
  • Never cache per-user pages in a shared cache without care, and watch out for stampedes on hot keys.

Next lesson: Logging, Email and Internationalization.