Skip to content
Tutorials Center
My learning

Topics

Lesson 9 of 78 ยท Pods

Container Images in Kubernetes: Tags, Digests and Pull Policy

Learn how Kubernetes names and pulls container images: tags vs digests, imagePullPolicy defaults, private registries with imagePullSecrets, updates and pull errors.

Every container in a Pod starts from an image, and the kubelet on the node pulls that image from a registry. Most "my Pod will not start" problems in the first week with Kubernetes are image problems: a typo in a tag, a private registry without credentials, a latest tag that quietly changed. Understanding how an image reference is read, when Kubernetes pulls and what the errors mean saves hours.

This lesson covers image names, tags and digests, the imagePullPolicy rules (and their surprising defaults), private registries, how to update a running workload, and how to read pull failures. There is no cluster on this site, so the manifests and kubectl commands are read-only. The two runnable examples are Python stand-ins for the parsing rules and for the retry behavior.

Anatomy of an image reference

TEXT
registry.example.com:5000 / team / web : v7 @ sha256:3f9a...
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ registry โ”€โ”€โ”€โ”˜  โ”” repository โ”˜ โ””tagโ”˜ โ””โ”€โ”€โ”€ digest โ”€โ”€โ”€โ”˜
  • Registry. The host that stores images. If the first part of the name has no dot or colon (and is not localhost), Kubernetes assumes the default registry of the runtime, normally Docker Hub (docker.io).
  • Repository. The path of the image inside the registry. On Docker Hub, a single word such as nginx means library/nginx.
  • Tag. A human-readable label such as 1.27 or v7. If you leave it out, it is latest.
  • Digest. The SHA-256 hash of the image manifest. It identifies exact content.

The stand-in below parses a few references and shows which pull policy Kubernetes would pick by default (more on that in a moment).

Python Reading image references

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

This is a simplified parser. The real one has more rules (for example, IPv6 hosts and normalization), but the shape is the same. Note the last two references: if both a tag and a digest are present, the digest wins and the tag is only documentation.

Tags are labels, digests are content

A tag is a pointer that the registry owner can move. nginx:1.27 today and nginx:1.27 next month can be different images, because maintainers rebuild tags to include security fixes. The tag latest is only a convention, and it moves with every release. That is convenient for experiments and dangerous for production, because two Pods created a week apart from the same manifest can run different code.

A digest is immutable. Pinning it makes the manifest reproducible:

YAML
containers:
  - name: web
    image: registry.example.com/team/web@sha256:3f9a0c5e...   # exact content

You can see which digest a running container actually uses:

Terminal
kubectl get pod web-7c9d8b5f6-abcde \
  -o jsonpath='{.status.containerStatuses[*].imageID}{"\n"}'

Practical guidance:

Situation Reference style
Learning, throwaway tests nginx or nginx:1.27 is fine
Production deployments A specific version tag, such as team/web:2.3.1, produced once by CI and never re-pushed
Strict reproducibility or supply-chain policy Tag plus digest, or digest only; automate bumps with a tool such as Renovate or your GitOps pipeline (GitOps)
Never latest in production manifests

Tag your images in CI with the Git commit or release number so that every build is traceable (Image Tagging Strategy).

imagePullPolicy

The imagePullPolicy field of a container decides when the kubelet contacts the registry:

Value Behavior
IfNotPresent Pull only if the node does not already have the image.
Always Ask the registry every time a container starts. If the local copy has the same digest, no layers are downloaded, but the check still needs the registry to be reachable.
Never Never pull. The image must already be on the node, or the container fails with ErrImageNeverPull.

If you do not set it, Kubernetes picks a default when the object is created:

  • no tag, or tag latest (and no digest): Always
  • any other tag, or a digest: IfNotPresent

That default is applied at creation and stored in the object. Changing the tag later, for example from latest to 1.27, does not change a policy that was already stored, so remove the field or set it explicitly when you edit manifests.

Consequences worth remembering:

  • With IfNotPresent and a reused tag, a node that already has the old image will keep running it. If you re-push web:v7 and expect new Pods to pick it up, they might not. Use a new tag per build.
  • With Always, a registry outage stops new Pods from starting, even if the image is cached on the node.
  • Never is for pre-loaded images, for example in local clusters: kind load docker-image web:dev or minikube image load web:dev put an image from your Docker daemon into the cluster's nodes (Running a Local Cluster).

Private registries: imagePullSecrets

A public image needs no credentials. A private one returns unauthorized until the kubelet presents them. You store registry credentials in a Secret of type kubernetes.io/dockerconfigjson and reference it from the Pod:

Terminal
kubectl create secret docker-registry regcred \
  --docker-server=registry.example.com \
  --docker-username=ci-reader \
  --docker-password='<token>' \
  --docker-email[email protected]
YAML
spec:
  imagePullSecrets:
    - name: regcred
  containers:
    - name: web
      image: registry.example.com/team/web:2.3.1

Adding the secret to every Pod spec is tedious. Attach it to the ServiceAccount instead, and every Pod that uses that account gets it automatically (Service Accounts):

Terminal
kubectl patch serviceaccount default \
  -p '{"imagePullSecrets": [{"name": "regcred"}]}'

Rules to remember: the Secret must be in the same namespace as the Pod, and credentials should be read-only tokens that only pull. Cloud providers offer alternatives: on EKS, GKE and AKS, nodes can authenticate to the provider's registry through their cloud identity, with no Secret (Secrets covers storing credentials safely).

Updating the image of a running workload

The image of a Pod cannot be changed in place for most fields (a container's image can be patched, and the kubelet restarts that container, but you should not do this to bare Pods). For managed workloads, edit the Deployment and let it roll:

Terminal
kubectl set image deployment/web web=registry.example.com/team/web:2.3.2
kubectl rollout status deployment/web
kubectl rollout undo deployment/web          # if the new version is bad

Or change the manifest and kubectl apply it, which is what GitOps does (Rolling Updates, Deployments). If you used a mutable tag and just need new Pods to pull again, kubectl rollout restart deployment/web recreates them, but with IfNotPresent they can still reuse a cached image. The fix is a new tag.

Reading pull errors

When the kubelet cannot get the image, the container waits in Waiting with one of these reasons. kubectl describe pod shows the details in Events.

Status Meaning Typical cause
ErrImagePull The pull just failed. Wrong name or tag, no network, no permission
ImagePullBackOff The kubelet is waiting before the next try. Same causes, repeated
InvalidImageName The reference cannot be parsed. Typo, uppercase letters in the repository
ErrImageNeverPull Policy is Never and the image is missing. Forgot to load the image
ImageInspectError The image is present but unusable. Corrupted image or an unsupported platform

The message text tells you which: manifest unknown (tag does not exist), pull access denied or unauthorized (credentials), no such host (registry address or DNS), no matching manifest for linux/arm64 (the image was built for another CPU architecture, see Multi-platform Images).

The kubelet retries with exponential back-off: it starts at 10 seconds and doubles up to 5 minutes. This stand-in plays that out, including the moment you push the missing image:

Python Image pull retries and back-off

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Notice that after you push the image, the Pod does not start immediately; it starts at the next retry. Deleting the Pod (so the controller recreates it) skips the wait. Debugging steps in general are in Troubleshooting, and the kubelet pulls images one at a time by default, so a Pod with several large images on a fresh node can be slow even without errors.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What does nginx mean as an image reference?

    With no registry, Docker Hub is assumed; a single-word repository means library/<name>; no tag means latest.
  2. 2. You did not set imagePullPolicy and use team/web:2.3.1. What default applies?

    The default is Always only for the latest tag (or no tag) without a digest. Any other tag or a digest defaults to IfNotPresent.
  3. 3. Why is it risky to re-push a different build to the same tag, such as web:v7, with IfNotPresent?

    IfNotPresent skips the pull when a tag exists locally, so nodes can disagree. Use immutable, unique tags (or digests).
  4. 4. A Pod shows ImagePullBackOff and the event says unauthorized. What do you check?

    unauthorized means the registry rejected the credentials. The pull Secret must exist in the Pod's namespace and be referenced by the Pod or its ServiceAccount.
  5. 5. Which reference is pinned to exact content?

    A digest identifies the exact image manifest; tags can be moved by whoever owns the repository.

Summary

  • An image reference is registry, repository, tag and/or digest; missing parts default to Docker Hub and latest, and a digest overrides the tag.
  • Use unique version tags (or digests) in production. Reusing a tag plus IfNotPresent makes nodes diverge.
  • imagePullPolicy defaults to Always for latest or no tag, otherwise IfNotPresent; Never is for preloaded images in local clusters.
  • Private registries need a docker-registry Secret in the Pod's namespace, referenced by imagePullSecrets on the Pod or its ServiceAccount.
  • ErrImagePull becomes ImagePullBackOff with retries that double up to five minutes; read the event message to see whether the tag, the credentials or the network is at fault.

Next, learn how Pods get their configuration in Environment Variables and the Downward API.