Skip to content
Tutorials Center
My learning

Topics

Lesson 76 of 111 ยท Registries and CI/CD

Running a Private Registry

Run your own Docker registry with the registry:2 image: push and pull, query the API, delete images, garbage-collect, add TLS and password auth, and know when not to.

Sooner or later someone asks, "Can we host our own images?" Maybe the code must not leave your network, maybe the CI server pulls the same base images hundreds of times a day, or maybe you just want a place to push test builds without paying for private repositories. Docker publishes the registry server itself as an image called registry:2, so a working private registry is one docker run away.

In this lesson you will start one, push and pull through it, talk to its HTTP API directly, delete an image and watch garbage collection give the disk space back, and then see what it takes to make it safe for more than your own laptop: TLS and authentication. It was all run on Docker 29 with the registry:2 image (the CNCF Distribution project, version 2.8.3 at the time of writing). The two things that need a real hostname or an account are marked read-only example. If you have not read Tags and Registries and Working with Registries, skim them first.

Start a registry in one command

CONSOLE
$ docker run -d --name registry -p 127.0.0.1:5000:5000 \
    -v registry-data:/var/lib/registry \
    -e REGISTRY_STORAGE_DELETE_ENABLED=true \
    registry:2

(In the lesson's verification run, port 18150 and the names tcd-b11-* were used; the commands are shown with the conventional port 5000.) Three choices are worth explaining:

  • -p 127.0.0.1:5000:5000 publishes the port only on the loopback interface. A registry with no authentication that listens on every network interface is an open file server for your images, and anyone who can push to it can replace the images your servers run.
  • -v registry-data:/var/lib/registry keeps the stored layers in a named volume. Without it, removing the container deletes every image (Volumes).
  • REGISTRY_STORAGE_DELETE_ENABLED=true allows deleting images through the API. It is off by default, which means images can be uploaded forever and never removed.

Every registry setting can be given as an environment variable: the name is the YAML path in upper case, joined with underscores (storage.delete.enabled becomes REGISTRY_STORAGE_DELETE_ENABLED). Check that it answers; the API root returns an empty JSON object and 200:

CONSOLE
$ curl -s -w ' [%{http_code}]\n' http://127.0.0.1:5000/v2/
{} [200]
$ curl -s http://127.0.0.1:5000/v2/_catalog
{"repositories":[]}

The catalog lists repositories, and it is empty because nothing has been pushed yet.

Push and pull

An image name beginning with 127.0.0.1:5000/ (or localhost:5000/) tells Docker to use this registry instead of Docker Hub. To push an existing image, give it a second name with the registry host in front, then push that name:

CONSOLE
$ docker tag alpine:3 127.0.0.1:5000/demo/alpine:1.0.0
$ docker tag alpine:3 127.0.0.1:5000/demo/alpine:latest
$ docker push 127.0.0.1:5000/demo/alpine:1.0.0
The push refers to repository [127.0.0.1:5000/demo/alpine]
7b229517ed22: Pushed
1.0.0: digest: sha256:c26612752c78b18faacd63c7d50f4592c2e7b0f25e0e20e82bd74e465c57ab0d size: 1022
$ docker push 127.0.0.1:5000/demo/alpine:latest | tail -1
latest: digest: sha256:c26612752c78b18faacd63c7d50f4592c2e7b0f25e0e20e82bd74e465c57ab0d size: 1022

Both tags have the same digest: they are two names for one manifest. (On Docker Desktop with the containerd image store you may also see an "Info -> Not all multiplatform-content is present" note. It means the local copy only has the layers for your own CPU architecture, so only that platform's manifest was pushed. See Multi-Platform Images.) Now ask the registry what it holds:

CONSOLE
$ curl -s http://127.0.0.1:5000/v2/_catalog
{"repositories":["demo/alpine"]}
$ curl -s http://127.0.0.1:5000/v2/demo/alpine/tags/list
{"name":"demo/alpine","tags":["latest","1.0.0"]}
$ curl -sI -H 'Accept: application/vnd.oci.image.manifest.v1+json' \
    http://127.0.0.1:5000/v2/demo/alpine/manifests/1.0.0 | grep -i docker-content-digest
Docker-Content-Digest: sha256:c26612752c78b18faacd63c7d50f4592c2e7b0f25e0e20e82bd74e465c57ab0d

The Accept header selects the manifest format; without a matching one the registry answers 404. Remove the local copies and pull back from the registry to prove the round trip:

CONSOLE
$ docker rmi 127.0.0.1:5000/demo/alpine:1.0.0 127.0.0.1:5000/demo/alpine:latest
$ docker pull 127.0.0.1:5000/demo/alpine:1.0.0
1.0.0: Pulling from demo/alpine
Digest: sha256:c26612752c78b18faacd63c7d50f4592c2e7b0f25e0e20e82bd74e465c57ab0d
Status: Downloaded newer image for 127.0.0.1:5000/demo/alpine:1.0.0
127.0.0.1:5000/demo/alpine:1.0.0

Docker treats localhost and 127.0.0.0/8 addresses as safe and accepts plain HTTP for them. For any other host name the daemon insists on HTTPS with a certificate it trusts. Do not "fix" that with insecure-registries in daemon.json on a real network: it turns off TLS checks for that host and lets anyone on the path read or modify what you pull. Use TLS (below).

Deleting images and garbage collection

Registry storage is content-addressed: layers (blobs) are stored once by digest and shared by every image that uses them. Because of that, the delete API removes only the manifest; the layers stay on disk until a separate garbage-collection run finds that nothing references them.

Deleting works by digest, not by tag. Delete the manifest we pushed:

CONSOLE
$ curl -s -o /dev/null -w '%{http_code}\n' -X DELETE \
    http://127.0.0.1:5000/v2/demo/alpine/manifests/sha256:c26612752c78b18faacd63c7d50f4592c2e7b0f25e0e20e82bd74e465c57ab0d
202
$ curl -s http://127.0.0.1:5000/v2/demo/alpine/tags/list
{"name":"demo/alpine","tags":null}

Both 1.0.0 and latest are gone, because they pointed at the same digest. That is the sharp edge of digest-based deletion: you cannot delete one tag of two that share an image. Delete the tag name by repointing it (push something else to it) or accept that all names for the manifest disappear. Disk usage did not change at all, which is why the second step matters:

CONSOLE
$ docker exec registry du -sh /var/lib/registry
3.8M	/var/lib/registry
$ docker exec registry registry garbage-collect --dry-run /etc/docker/registry/config.yml
demo/alpine

0 blobs marked, 3 blobs and 0 manifests eligible for deletion
blob eligible for deletion: sha256:7b229517ed22...
blob eligible for deletion: sha256:b92841763c5c...
blob eligible for deletion: sha256:c26612752c78...

--dry-run only reports. The three blobs are the layer, the image configuration and the manifest itself (a manifest is also stored as a blob). Run it for real:

CONSOLE
$ docker exec registry registry garbage-collect /etc/docker/registry/config.yml
$ docker exec registry du -sh /var/lib/registry
96.0K	/var/lib/registry

From 3.8 MB to 96 KB. The config path /etc/docker/registry/config.yml is where the registry:2 image keeps its default configuration (if the command complains about a missing file, look inside your image for its config path). Two cautions:

  • Garbage collection is not safe to run while someone is pushing. A layer uploaded a moment ago but not yet referenced by a manifest looks like garbage. Schedule it in a quiet window, or stop the registry (or make it read-only) while it runs.
  • It deletes only unreferenced blobs. To also remove manifests that have no tag, add --delete-untagged.

Notice that the repository name demo/alpine still shows up in the catalog after the cleanup. Empty repositories remain listed; the catalog is not a reliable "what images exist" source once you start deleting.

Make it safe: TLS and authentication

A registry that other machines use needs two things the quick start skipped: encryption (so images and credentials are not readable on the wire) and authentication (so not everyone can push). The registry:2 image supports both through configuration. This Compose file combines them. You need a certificate whose name or IP address matches the host clients use; the certificates created in Securing the Docker Daemon work for a test:

YAML
name: reg
services:
  registry:
    image: registry:2
    restart: unless-stopped
    ports:
      - "127.0.0.1:5001:5000"
    environment:
      REGISTRY_HTTP_TLS_CERTIFICATE: /certs/server-cert.pem
      REGISTRY_HTTP_TLS_KEY: /certs/server-key.pem
      REGISTRY_AUTH: htpasswd
      REGISTRY_AUTH_HTPASSWD_REALM: Registry
      REGISTRY_AUTH_HTPASSWD_PATH: /auth/htpasswd
      REGISTRY_STORAGE_DELETE_ENABLED: "true"
    volumes:
      - registry-data:/var/lib/registry
      - ./server-cert.pem:/certs/server-cert.pem:ro
      - ./server-key.pem:/certs/server-key.pem:ro
      - ./htpasswd:/auth/htpasswd:ro
volumes:
  registry-data:

The password file uses bcrypt. The registry image does not include the htpasswd tool, so create the file with a throwaway container (-B selects bcrypt, -b takes the password from the command line, -n prints instead of editing a file; the sample password is made up):

CONSOLE
$ docker run --rm alpine:3 sh -c 'apk add --no-cache apache2-utils >/dev/null 2>&1 && htpasswd -Bbn alice s3cret-demo' > htpasswd
$ cut -c1-20 htpasswd
alice:$2y$05$mgcELt2

Start it and look at the log lines that confirm TLS is on:

CONSOLE
$ docker compose up -d
 Container reg-registry-1 Created
 Container reg-registry-1 Started
$ docker logs reg-registry-1 2>&1 | grep -o 'msg="[^"]*"' | sed -n '2p;4p'
msg="restricting TLS version to tls1.2 or higher"
msg="listening on [::]:5000, tls"

Now the API behaves differently for three kinds of callers. A caller with no credentials is refused; one with a wrong password is refused; the right user gets in. (The --cacert option makes curl trust the private certificate authority; on Windows, curl may also need --ssl-no-revoke because it cannot reach a revocation server for a private CA.)

CONSOLE
$ curl -s --cacert ca.pem -w ' [%{http_code}]\n' https://127.0.0.1:5001/v2/
{"errors":[{"code":"UNAUTHORIZED","message":"authentication required","detail":null}]}
 [401]
$ curl -s --cacert ca.pem -u alice:wrong -w ' [%{http_code}]\n' https://127.0.0.1:5001/v2/_catalog
{"errors":[{"code":"UNAUTHORIZED","message":"authentication required","detail":[{"Type":"registry","Class":"","Name":"catalog","Action":"*"}]}]}
 [401]
$ curl -s --cacert ca.pem -u alice:s3cret-demo -w ' [%{http_code}]\n' https://127.0.0.1:5001/v2/_catalog
{"repositories":[]} [200]
$ curl -si --cacert ca.pem https://127.0.0.1:5001/v2/ | grep -i www-authenticate
Www-Authenticate: Basic realm="Registry"

Without --cacert, the client refuses to talk because it cannot trust the certificate. The same applies to Docker: asking the daemon to use this registry without a login fails at the authentication step:

CONSOLE
$ docker pull 127.0.0.1:5001/demo/alpine:1.0.0
Error response from daemon: failed to resolve reference "127.0.0.1:5001/demo/alpine:1.0.0": pull access denied, repository does not exist or may require authorization: authorization failed: no basic auth credentials

Everything beyond this point needs a real host name and an account, so it is a read-only example. On a server named registry.example.com:

Terminal
# on every Docker host that will use the registry (Linux)
sudo mkdir -p /etc/docker/certs.d/registry.example.com
sudo cp ca.pem /etc/docker/certs.d/registry.example.com/ca.crt

docker login registry.example.com -u alice
docker tag web:1.0.0 registry.example.com/team/web:1.0.0
docker push registry.example.com/team/web:1.0.0

The directory name under certs.d must match the host (and port, if not 443, as registry.example.com:5000). With a certificate from a public CA (for example Let's Encrypt) no extra step is needed.

Operating it for real

  • Backups. Everything is in the storage backend. Back up the volume (Backup and Restore) and your htpasswd file. For cloud storage the registry can write directly to S3, Azure Blob or Google Cloud Storage instead of local disk.
  • A reverse proxy such as nginx or Traefik in front of the registry is common: it terminates TLS, applies rate limits, and handles large uploads. Raise the maximum body size, because layers can be gigabytes.
  • Pull-through cache. Setting proxy.remoteurl: https://registry-1.docker.io turns a registry into a mirror of Docker Hub, which avoids rate limits and speeds up CI. A mirror serves only pulls; it cannot also accept your pushes.
  • Authentication limits. htpasswd is a single flat user list with no per-repository permissions. Anything more serious, such as teams, roles, scanning or a web interface, is what products like Harbor and the registries built into GitLab and cloud platforms offer.
  • Monitoring. The registry exposes a /debug/health and Prometheus metrics endpoint when configured, and logs to stdout, so docker logs and a log driver cover the basics.

The plain registry:2 has no web interface and no built-in image scanning or retention policy. If more than a handful of people depend on it, evaluate Harbor or a managed registry instead of building those features around the bare server.

When to use a private registry (and when not to)

Situation Choice
Local experiments, testing a build on another machine, CI cache registry:2 on loopback is perfect
Air-gapped network, data cannot leave the building Self-hosted (Harbor or registry:2 with TLS and auth)
Small team using GitHub or GitLab The platform's registry: zero setup, permissions follow the repository
Production on AWS, Google Cloud or Azure The provider's registry in the same region
A single image to move once docker save and docker load (Save, Load, Export, Import)

Self-hosting is cheap to start and expensive to keep running: you own the backups, upgrades, certificate renewals and garbage collection. Choose it for a reason, not by default.

Clean up

When you are done experimenting, remove what you created so that the volume does not keep your test images forever:

CONSOLE
$ docker rm -f registry
$ docker volume rm registry-data
$ docker rmi 127.0.0.1:5000/demo/alpine:1.0.0

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Why does the quick-start command publish the port as 127.0.0.1:5000:5000 instead of 5000:5000?

    A registry without authentication that listens on all interfaces lets anyone on the network read and replace your images. Binding to loopback avoids that.
  2. 2. You delete an image through the registry API, but the disk usage does not change. Why?

    Blobs are shared and content-addressed. Garbage collection finds blobs that no manifest references and removes them.
  3. 3. Two tags (1.0.0 and latest) point to the same digest. What happens when you delete the manifest by that digest?

    Deletion is by digest, and tags are only pointers to it. The tag list for the repository became empty in the example.
  4. 4. Why is running garbage collection while someone pushes an image dangerous?

    A layer uploaded moments ago but not yet tied to a pushed manifest looks unreferenced and can be removed, breaking the push.
  5. 5. A Docker host must pull from registry.example.com, which uses a certificate from your private CA. What should you do?

    Trusting the CA keeps TLS verification on. insecure-registries disables the checks and exposes pulls to tampering.

Summary

  • registry:2 is the Docker registry server as an image; run it with a volume and bind it to 127.0.0.1 unless it has TLS and authentication.
  • Name an image host:port/repo:tag to push to it; the registry's HTTP API (/v2/_catalog, /v2/<repo>/tags/list) lets you inspect it with curl.
  • Deletion works by digest, removes every tag pointing at it, and frees no space until registry garbage-collect runs (not while pushes are in progress).
  • Enable TLS with REGISTRY_HTTP_TLS_CERTIFICATE and REGISTRY_HTTP_TLS_KEY, and basic auth with an htpasswd file in bcrypt format.
  • Do not use insecure-registries outside loopback; install the CA under /etc/docker/certs.d/<host>/ca.crt.
  • For permissions, scanning, replication or a UI, use Harbor or the built-in registry of your platform instead.

Next: Tagging and Versioning Strategies