Skip to content
Tutorials Center
My learning

Topics

Lesson 78 of 111 ยท Registries and CI/CD

Docker in CI

Build, test and push Docker images from CI: test stages, tags from the commit SHA, registry layer caching, matrix builds, and GitHub Actions and GitLab CI examples.

On your laptop, docker build is something you run when you feel like it. In a team, the image that matters is the one built by a CI pipeline: a script that runs on a clean machine for every commit, runs the tests, and produces an image tagged so you can always trace it back to the code. Nobody's laptop, cache or forgotten local change leaks into it.

This lesson covers the moving parts of a Docker CI job: running tests inside the build, tagging images from the commit, pushing to a registry, and keeping builds fast on runners that start from nothing every time. The Docker commands below were run against a real local registry (registry:2), so their output is genuine. The GitHub Actions and GitLab CI files are read-only examples: this page cannot run a CI service, so they are written from the documented behavior of those tools and you should adapt them to your versions.

What a CI job does

Almost every Docker pipeline has the same shape, whatever CI product runs it:

TEXT
 git push
    |
    v
 +-----------+   +-----------+   +--------------+   +-----------+
 | checkout  |-->| build +   |-->| tag with     |-->| push to   |
 | the code  |   | run tests |   | SHA / version|   | registry  |
 +-----------+   +-----------+   +--------------+   +-----------+
                      |                                   |
                  fails? stop                       deploy picks up
                  here, nothing is pushed           the tagged image

Two ideas make this reliable. First, the image is the artifact: you build it once, and the same bytes travel through test, staging and production. Second, a failing test stops the pipeline before anything is pushed, so a registry never contains an image that did not pass.

Run the tests inside the build

The simplest way to test with Docker is to make the tests a stage of a multi-stage build. Here is a Node.js app (the same one from Containerizing a Node.js App) with a test stage that has the development dependencies and a lean runtime stage that does not:

DOCKERFILE
# syntax=docker/dockerfile:1
ARG NODE_VERSION=24
FROM node:${NODE_VERSION}-alpine AS base
WORKDIR /app
COPY package.json package-lock.json ./

FROM base AS deps-prod
RUN npm ci --omit=dev

FROM base AS deps-all
RUN npm ci

FROM deps-all AS test
ENV NODE_ENV=test
COPY src ./src
COPY test ./test
RUN node --test

FROM node:${NODE_VERSION}-alpine AS runtime
ENV NODE_ENV=production
WORKDIR /app
COPY --from=deps-prod --chown=node:node /app/node_modules ./node_modules
COPY --chown=node:node package.json ./
COPY --chown=node:node src ./src
USER node
CMD ["node", "src/server.js"]

The test stage ends in RUN node --test. A RUN that exits non-zero fails the build, so a failing test fails the job with a normal exit code, which is exactly what every CI system looks for. Select the stage with --target:

CONSOLE
$ docker build --target test --progress=plain -t app-test .
...
#13 [test 3/3] RUN node --test
#13 0.835 โœ” addition still works (2.45ms)
#13 0.852 โœ” NODE_ENV is test inside the test stage (0.36ms)
#13 0.878 โ„น tests 2
#13 0.878 โ„น pass 2
#13 0.878 โ„น fail 0
#13 DONE 1.2s

Now break a test on purpose and build again:

CONSOLE
$ docker build --target test --progress=plain -t app-test .
...
#13 0.854 โœ– addition still works (15.31ms)
#13 0.933   AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
...
ERROR: failed to build: failed to solve: process "/bin/sh -c node --test" did not complete successfully: exit code: 1
$ echo $?
1

Exit code 1, no image tagged app-test, and the pipeline stops. Because the runtime stage is a separate branch, the production image never contains test files or dev dependencies, and BuildKit skips the test stage entirely when you build the default target (it is not needed for the last stage).

Order the CI steps so the cheap checks come first: lint, then unit tests, then the image build. Run the test stage with --target test first, and build the runtime image only if it passes. Both builds share the cache, so the second one costs almost nothing.

You can also run tests against a built image, with docker run --rm app-image npm test, or against a whole system with Compose (docker compose up --abort-on-container-exit). The stage approach is the most portable because it needs nothing but docker build.

Tagging from the commit

A CI image needs a name that says exactly which commit produced it. The convention is the short Git SHA, plus a moving tag for the branch:

Tag Example Meaning
Commit SHA app:3f9a2c1 Immutable. Points at exactly one commit. Deploy this.
Branch app:main Moves with every push to main. Handy for staging.
Release app:v1.0.0 Created when a Git tag is pushed.

The tagging and versioning strategies lesson covers the reasoning. Here is the full flow against a local registry on port 18150. The CI system normally gives you the SHA in a variable; the value below is a stand-in:

CONSOLE
$ REG=127.0.0.1:18150/team/app
$ SHA=3f9a2c1
$ docker buildx build --target runtime \
    -t $REG:$SHA -t $REG:main --push .
...
#15 pushing layers 0.4s done
#15 pushing manifest for 127.0.0.1:18150/team/app:3f9a2c1@sha256:000bede8...
$ curl -s http://127.0.0.1:18150/v2/team/app/tags/list
{"name":"team/app","tags":["3f9a2c1","main"]}

--push uploads straight from the builder, so there is no separate docker push step, and one build can carry several -t tags. When you cut a release, do not rebuild: promote the image that already passed everything by giving it another tag in the registry:

CONSOLE
$ docker buildx imagetools create -t $REG:v1.0.0 $REG:3f9a2c1
#1 0.117 pushing sha256:000bede8... to 127.0.0.1:18150/team/app:v1.0.0
$ curl -s http://127.0.0.1:18150/v2/team/app/tags/list
{"name":"team/app","tags":["3f9a2c1","main","v1.0.0"]}

v1.0.0 and 3f9a2c1 now point to the same digest. The bytes you tested are the bytes you ship. Setting up a registry of your own is covered in Running a Private Registry.

Caching on throwaway runners

Hosted CI runners start empty, so the layer cache from your last build is gone and every job would download base images and reinstall dependencies. The fix is to store the cache somewhere that outlives the runner. BuildKit can export it to a registry and import it again:

CONSOLE
$ docker buildx build --target test \
    --cache-from type=registry,ref=$REG:buildcache \
    --cache-to   type=registry,ref=$REG:buildcache,mode=max .
#7 importing cache manifest from 127.0.0.1:18150/team/app:buildcache
#7 ERROR: failed to configure registry cache importer: ...:buildcache: not found
...
#9 sending cache export 5.7s done

The first run prints a not found error while importing, because the cache image does not exist yet. That error is not fatal: the build carries on and then exports the cache. The next run finds it, and the log shows CACHED for every unchanged step. mode=max exports the layers of all stages (including the test stage), not just the final image, which is what you want for multi-stage Dockerfiles.

The cache lives in the registry under an ordinary tag (buildcache here), so it appears in the tag list next to your real images:

CONSOLE
$ curl -s http://127.0.0.1:18150/v2/team/app/tags/list
{"name":"team/app","tags":["3f9a2c1","buildcache","main"]}

Cache backends you will meet:

Backend Flag Good for
Registry type=registry,ref=... Any CI, shared by all jobs and branches
GitHub Actions cache type=gha GitHub-hosted runners, no registry needed
Local directory type=local,dest=... A directory your CI saves and restores itself
Inline --cache-to type=inline Cache metadata stored in the pushed image itself (only the final stage)

Build performance is a topic of its own, with measurements, in Build Performance.

Matrix builds

A matrix runs the same job for several values, for example every Node.js version you support. With a build argument it is a one-line change per entry. This loop is what a CI matrix does for you in parallel:

CONSOLE
$ for v in 22 24; do
>   docker build --target test --build-arg NODE_VERSION=$v --progress=plain .
> done
...
#13 0.641 running on Node 24
#13 0.697 โ„น pass 3
...
#13 0.792 # running on Node 22
#13 0.811 ok 3 - reports its runtime

The ARG NODE_VERSION before the first FROM is what makes this possible (see FROM and RUN). Note that the two runs print results in different styles: Node 22's test runner writes TAP when its output is not a terminal, while Node 24 writes the readable format. Real pipelines should pick a reporter explicitly so logs look the same everywhere.

A GitHub Actions workflow (read-only example)

GitHub Actions has official Docker actions for exactly this flow. The version numbers below were current when this was written; check the Marketplace for newer ones.

YAML
name: ci
on:
  push:
    branches: [main]
    tags: ["v*"]
  pull_request:

permissions:
  contents: read
  packages: write

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: docker/setup-buildx-action@v3
      - name: Run the tests in the test stage
        uses: docker/build-push-action@v6
        with:
          context: .
          target: test
          push: false
          cache-from: type=gha
          cache-to: type=gha,mode=max

  publish:
    needs: test
    if: github.event_name != 'pull_request'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: docker/setup-buildx-action@v3
      - uses: docker/login-action@v3
        with:
          registry: ghcr.io
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}
      - id: meta
        uses: docker/metadata-action@v5
        with:
          images: ghcr.io/${{ github.repository }}
          tags: |
            type=sha
            type=ref,event=branch
            type=semver,pattern={{version}}
      - uses: docker/build-push-action@v6
        with:
          context: .
          target: runtime
          push: true
          tags: ${{ steps.meta.outputs.tags }}
          labels: ${{ steps.meta.outputs.labels }}
          cache-from: type=gha
          cache-to: type=gha,mode=max

Notice what is not there: no password typed into the file. The login uses a token that CI provides as a secret, and pull requests build and test but never push (push: false, and the publish job is skipped). That second rule matters: code from a fork should not be able to publish to your registry.

A GitLab CI file (read-only example)

GitLab CI usually builds with a Docker-in-Docker service or with a daemonless builder. This version uses the docker:dind service. The risks of that approach are discussed in Docker in Docker and Socket Access.

YAML
stages: [test, publish]

variables:
  DOCKER_TLS_CERTDIR: "/certs"
  IMAGE: $CI_REGISTRY_IMAGE

default:
  image: docker:cli
  services:
    - docker:dind

test:
  stage: test
  script:
    - docker build --target test .

publish:
  stage: publish
  rules:
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
  script:
    - echo "$CI_REGISTRY_PASSWORD" | docker login -u "$CI_REGISTRY_USER" --password-stdin "$CI_REGISTRY"
    - docker build --target runtime -t "$IMAGE:$CI_COMMIT_SHORT_SHA" -t "$IMAGE:latest" .
    - docker push "$IMAGE:$CI_COMMIT_SHORT_SHA"
    - docker push "$IMAGE:latest"

Here the daemon is brand new for every job, so nothing is cached unless you add --cache-from/--cache-to as shown earlier. --password-stdin keeps the password out of the process list and your shell history.

CI pitfalls professionals avoid

  • Secrets in build args or layers. Anything in a RUN, ENV or ARG can end up in the image history. Use BuildKit secrets and the CI system's secret store.
  • Pushing from pull requests. Only trusted branches and tags should publish.
  • Deploying latest. Deploy the SHA or digest, so a rollback is a one-line change.
  • A different image in each environment. Rebuilding for production can produce different dependencies. Build once, promote the same digest.
  • No scanning or pinning. Add a vulnerability scan and pin base images; see Image Security and Scanning.
  • Ignoring the build context. A missing .dockerignore makes every CI run upload .git and node_modules; see Build Context and .dockerignore.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Why put the tests in a separate stage of the Dockerfile and build it with --target test?

    A RUN step that exits non-zero fails the build, which CI systems read as a failed job. The runtime stage is a separate branch, so test files and dev dependencies never reach the production image.
  2. 2. A teammate wants to release v1.0.0 and proposes rebuilding the image from the tag. What is the better approach?

    A rebuild can pull different dependency versions. Promoting means adding another tag to the same digest, so the exact bytes that passed the tests are the ones shipped.
  3. 3. On the very first CI run, the log shows failed to configure registry cache importer: ... not found. What does it mean?

    --cache-from points at a tag that is created by the first --cache-to. A missing cache is not fatal. Later runs import it and show CACHED steps.
  4. 4. Which tag is the best one to deploy to production from a CI pipeline?

    latest and main move whenever someone pushes. A SHA tag points at one commit forever, so you know what is running and can roll back by deploying the previous SHA.
  5. 5. What does mode=max do in --cache-to type=registry,ref=...,mode=max?

    The default mode min exports only the layers of the final image. With multi-stage builds that would skip the dependency and test stages, so use max to cache them too.

Summary

  • A CI image is built by a clean runner for every commit and should be the same artifact that is tested, stored and deployed.
  • Put tests in a test stage and run docker build --target test: a failing test fails the build with exit code 1 and nothing is pushed.
  • Tag with the commit SHA (immutable), the branch name (moving) and release tags; promote an existing image with docker buildx imagetools create instead of rebuilding.
  • docker buildx build --push builds and uploads in one step and accepts several -t flags.
  • Runners start empty, so export the layer cache with --cache-to and import it with --cache-from (registry, gha, local directory). A missing cache on the first run is harmless; use mode=max for multi-stage builds.
  • Matrix builds pass a build argument, such as the Node.js version, to the same Dockerfile.
  • Keep secrets in the CI secret store, never publish from pull requests, and deploy by SHA or digest rather than latest.

Next: Build Performance