Skip to content
Tutorials Center
My learning

Topics

Lesson 26 of 111 ยท Dockerfiles

The Build Cache

Learn how the Docker build cache decides what to reuse, why instruction order matters, how to copy dependencies first, and how to bust or skip the cache safely.

Every time you run docker build, Docker has a choice: do each step again, or reuse the result from the last time it ran the same step. Reusing is the build cache, and it is the difference between a rebuild that takes half a second and one that takes a minute. On a team or in CI, where images are built hundreds of times a day, good cache use is one of the cheapest speed-ups available.

The cache is mostly automatic, but it follows strict rules. If you do not know them, you will write Dockerfiles that reinstall every dependency whenever you fix a typo. In this lesson you will learn how cache keys work, why the order of instructions matters so much, the "dependencies first" pattern, and how to deliberately bypass the cache when you need fresh results. All outputs below come from real builds of a small Python app.

How the cache works

Each instruction in a Dockerfile produces a layer (see Layers and the Union Filesystem). For every instruction, the builder computes a cache key and looks for a stored layer with the same key. If it finds one, the step is skipped and the log says CACHED.

What goes into the key depends on the instruction:

Instruction The cache key includes
RUN The exact command text, plus the state of all previous layers and any build args in scope. The command is not executed to check whether the result would be the same.
COPY / ADD The content (checksum) of the files being copied, not their timestamps.
ENV, WORKDIR, USER, CMD... The instruction text and the parent layer.
FROM The base image digest (the tag is resolved to a digest).

The most important rule is this: if one step is not served from the cache, every step after it is rebuilt too. The cache is a chain. A layer's key depends on its parent's, so a change anywhere invalidates everything below it.

TEXT
 FROM python:3.13-slim        cached   (same base image)
 WORKDIR /app                 cached
 COPY requirements.txt .      cached   (file content unchanged)
 RUN pip install ...          cached
 COPY . .                  <- app.py changed: rebuilt
 CMD ["python", "app.py"]  <- rebuilt too (cheap)

Seeing it in practice is the best way to understand it. Use --progress=plain to get a readable log that shows CACHED markers; the default progress display redraws in place and hides them.

A slow Dockerfile

Here is the Dockerfile most beginners write first. The project has app.py and a requirements.txt that pins the requests library.

DOCKERFILE
FROM python:3.13-slim
WORKDIR /app
COPY . .
RUN pip install --no-cache-dir -r requirements.txt
CMD ["python", "app.py"]

Build it twice without changing anything:

CONSOLE
$ docker build --progress=plain -t slow-app .
...
#6 [2/4] WORKDIR /app
#7 [3/4] COPY . .
#8 [4/4] RUN pip install --no-cache-dir -r requirements.txt
#8 DONE 8.4s
$ docker build --progress=plain -t slow-app .
...
#6 [2/4] WORKDIR /app
#6 CACHED
#7 [3/4] COPY . .
#7 CACHED
#8 [4/4] RUN pip install --no-cache-dir -r requirements.txt
#8 CACHED

The first build ran pip install, which took 8.4 seconds on this machine. The second build was served entirely from the cache. So far so good. Now change one line in app.py (the code, not the dependencies) and rebuild:

CONSOLE
$ sed -i 's/v1/v2/' app.py
$ docker build --progress=plain -t slow-app .
...
#6 [2/4] WORKDIR /app
#6 CACHED
#7 [3/4] COPY . .
#7 DONE 0.0s
#8 [4/4] RUN pip install --no-cache-dir -r requirements.txt
#8 DONE 11.7s

COPY . . copies the whole project, so changing app.py changes its checksum. That invalidates the layer, and the chain rule then forces pip install to run again, even though requirements.txt did not change. Eleven seconds to learn nothing new. Multiply it by every code change, every developer and every CI run.

The fast version: copy dependencies first

The fix is to order the instructions from least often changed to most often changed, and to copy only what each step needs:

DOCKERFILE
FROM python:3.13-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
CMD ["python", "app.py"]

Now pip install only depends on requirements.txt. After the first build, edit app.py and rebuild:

CONSOLE
$ docker build --progress=plain -t fast-app .
...
#6 [2/5] WORKDIR /app
#6 CACHED
#7 [3/5] COPY requirements.txt .
#7 CACHED
#8 [4/5] RUN pip install --no-cache-dir -r requirements.txt
#8 CACHED
#9 [5/5] COPY . .
#9 DONE 0.1s

The slow step was skipped, and only the final COPY . . of the changed code ran (a tenth of a second here). Only when the dependency list changes does the install run again:

CONSOLE
$ printf 'requests==2.32.3\nidna==3.10\n' > requirements.txt
$ docker build --progress=plain -t fast-app .
...
#7 [3/5] COPY requirements.txt .
#7 DONE 0.1s
#8 [4/5] RUN pip install --no-cache-dir -r requirements.txt
#8 DONE 11.7s
$ docker run --rm fast-app
app v2, requests 2.32.3

This pattern is the most valuable cache habit you can learn, and it works in every ecosystem:

Language Copy first Then install Then copy the rest
Node.js package.json, package-lock.json npm ci COPY . .
Python requirements.txt (or pyproject.toml and lock file) pip install -r COPY . .
Go go.mod, go.sum go mod download COPY . .
PHP composer.json, composer.lock composer install COPY . .
Java (Maven) pom.xml mvn dependency:go-offline COPY src ./src

The golden rule: put what changes rarely at the top (base image, system packages, dependency manifests, dependency installation) and what changes often at the bottom (your source code). Read your Dockerfile from top to bottom and ask for each line, "how often does its input change?"

COPY checks content, not timestamps

A common worry is "does touch or a git checkout, which updates timestamps, ruin my cache?" It does not. The key for COPY is a checksum of the content (plus a few attributes such as permissions). Let us verify it:

DOCKERFILE
FROM alpine:3.22
COPY data.txt /data.txt
RUN date +%s > /built-at.txt
RUN echo "step three"
CONSOLE
$ docker build -t ts-demo .
$ docker run --rm ts-demo cat /built-at.txt
1792765903
$ touch data.txt
$ docker build --progress=plain -t ts-demo .
...
#6 [2/4] COPY data.txt /data.txt
#6 CACHED
#7 [3/4] RUN date +%s > /built-at.txt
#7 CACHED
#8 [4/4] RUN echo "step three"
#8 CACHED
$ docker run --rm ts-demo cat /built-at.txt
1792765903

The file's timestamp changed but not its content, so everything stayed cached and /built-at.txt still holds the old time. Now actually change the content:

CONSOLE
$ echo data2 > data.txt
$ docker build --progress=plain -t ts-demo .
...
#6 [2/4] COPY data.txt /data.txt
#7 [3/4] RUN date +%s > /built-at.txt
#8 [4/4] RUN echo "step three"
$ docker run --rm ts-demo cat /built-at.txt
1792765920

All three steps ran, and the timestamp is new. This example also shows the second half of the rule: the cache never executes your command to see whether the output would differ. The date command's result is frozen in the layer until something above it changes.

Similarly, editing the text of a RUN line invalidates it. Changing RUN echo "step three" to RUN echo "step 3" re-runs just that step (the earlier COPY and RUN date stay CACHED).

Pitfall: stale results from commands that fetch things

Because the cache trusts the command text, any RUN that fetches changing data can quietly serve an old answer:

DOCKERFILE
RUN apt-get update
RUN apt-get install -y curl

If these are two separate instructions, the first (apt-get update) can be cached from last month while the second runs fresh and asks for a package that the stale package list no longer has, which fails with a 404. The cure is to do both in a single RUN so they are cached and invalidated together (see FROM and RUN):

DOCKERFILE
RUN apt-get update && apt-get install -y --no-install-recommends curl \
    && rm -rf /var/lib/apt/lists/*

The same problem hits RUN git clone ..., RUN curl .../latest | sh and similar steps. They produce today's result once, and the cache keeps that result forever. Either pin an explicit version in the command (which changes the text and so busts the cache when you bump it) or bust the cache on purpose, as below.

Busting the cache on purpose

Sometimes you do want fresh results. You have several tools, from broad to precise.

1. --no-cache: ignore the cache for the whole build.

CONSOLE
$ docker build --no-cache -t fast-app .

Every instruction that does real work runs again (pip install was DONE 9.1s instead of CACHED). Use it to rule out cache problems, or for a scheduled "fresh" build that picks up security patches. It is slow, so do not make it a habit.

2. --pull: check the registry for a newer version of the base image first. If python:3.13-slim has been updated since you last pulled it, the new digest changes the FROM key and the whole chain rebuilds. Without --pull, Docker uses the copy already on your machine.

3. A build argument placed right before the step. Pass a changing value, and the RUN after it is no longer cached:

DOCKERFILE
FROM alpine:3.22
ARG CACHEBUST=0
RUN echo "fetching latest" && date +%s > /fetched.txt
CONSOLE
$ docker build --progress=plain -t bust .
...
#5 [2/2] RUN echo "fetching latest" && date +%s > /fetched.txt
#5 CACHED
$ docker build --progress=plain --build-arg CACHEBUST=$(date +%s) -t bust .
...
#5 [2/2] RUN echo "fetching latest" && date +%s > /fetched.txt

Position matters: the ARG must come before the instruction you want to refresh, because only steps after a changed ARG are invalidated (ENV and ARG shows the same effect). If you put it after, the earlier step stays cached.

4. --no-cache-filter <stage>: skip the cache for one named stage of a multi-stage build and keep it for the others.

DOCKERFILE
FROM alpine:3.22 AS deps
RUN echo "deps stage" && date +%s > /d.txt
FROM alpine:3.22 AS app
RUN echo "app stage" && date +%s > /a.txt
CONSOLE
$ docker build --progress=plain --no-cache-filter app --target app -t nf .
...
#5 [app 2/2] RUN echo "app stage" && date +%s > /a.txt

The app step ran fresh even though a cached copy existed. (Multi-Stage Builds explains stages.)

What else breaks the cache by accident

  • A noisy build context. COPY . . includes everything in the directory. If a log file, a .git folder or a build output directory changes on every run, the COPY layer is rebuilt every time, and everything below it with it. Use a .dockerignore file (Build Context and .dockerignore).
  • Generated files in the context. A version.txt or a timestamp file that your tooling rewrites before each build changes the checksum. Copy such files in the very last step, or generate them in the Dockerfile.
  • An ARG declared at the top whose value changes on every build (a commit SHA, a build number). It invalidates every RUN after it. Declare it as late as possible, or pass it only into the final label step.
  • Reordering or whitespace. Moving lines changes keys. Changing only a comment does not (comments are not instructions).
  • A different builder or machine. The cache lives on the machine that did the build. A fresh CI runner starts empty. Sharing a cache between runs needs an explicit setup (see Docker in CI and Build Performance).

The cache can also hide problems. If a build "works on my machine" only because a cached apt-get update is old, or a cached RUN downloaded a file that no longer exists upstream, a clean machine will fail. Run a --no-cache build in CI now and then.

Seeing and managing the cache

The cache is stored by BuildKit on the machine running the build. You can see how much space it uses:

CONSOLE
$ docker buildx du
...
Reclaimable:	21.52GB
Total:		21.52GB

(The numbers depend entirely on your history.) Watch the "Build Cache" row of docker system df too. To free the space you can run docker builder prune, but note that it removes cache for all builds on the daemon, not only one project, and the next builds will be slow once. Cleaning Up covers filters such as --filter until=24h, which keeps recent entries.

Finally, remember that a cache hit only skips work; it never changes the result. You get the identical image either way, as long as your Dockerfile is deterministic.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Your Dockerfile has COPY . . followed by RUN npm ci. You change one line of source code. What happens on the next build?

    COPY . . includes the changed file, so its checksum changes. Every later step is rebuilt because the cache is a chain. Copy only the manifest files first to avoid this.
  2. 2. Which ordering gives the best cache use for a typical app?

    Put rarely changing inputs first. The expensive install then depends only on the manifest and is reused when only code changes.
  3. 3. You run touch data.txt (no content change) and rebuild. What happens to COPY data.txt /data.txt?

    COPY and ADD use a checksum of the content, so updating only the timestamp keeps the cache valid.
  4. 4. A RUN git clone https://example.com/tools.git step always returns the old code even though the repository changed. Why?

    The cache key for RUN is the command text and the parent layer. Docker does not run the command to see whether the result changed. Pin a version, change the text, or bust the cache.
  5. 5. You want to rebuild just one named stage fresh while keeping the cache for others. Which option do you use?

    --no-cache-filter takes a stage name and disables the cache for just that stage. --no-cache disables it for everything, and --pull only refreshes the base image.

Summary

  • Each Dockerfile instruction gets a cache key. A hit prints CACHED; a miss rebuilds that step and every step after it.
  • RUN is keyed by its command text (the command is not re-executed to check), COPY/ADD by file content (not timestamps), FROM by the base image digest.
  • Order instructions from least to most frequently changing. Copy dependency manifests, install dependencies, then copy the source.
  • Commands that fetch changing data (apt-get update, git clone, curl | sh) can serve stale results from the cache: combine them correctly in one RUN, pin versions, or bust the cache.
  • Bust the cache with --no-cache (everything), --pull (new base image), a changing ARG placed right before the step, or --no-cache-filter <stage>.
  • Keep the build context clean with .dockerignore, and declare volatile ARGs as late as possible.
  • Use --progress=plain to read cache hits; docker buildx du shows cache size; pruning affects every build on the machine.

Next: Multi-Stage Builds