Skip to content
Tutorials Center
My learning

Topics

Lesson 17 of 78 ยท Workloads

Rolling Updates and Rollbacks

Understand how a Deployment replaces pods gradually with maxSurge and maxUnavailable, how to pause, inspect and roll back a rollout, and how history is kept.

Shipping a new version should not mean a maintenance window. A Deployment's default strategy, RollingUpdate, starts pods of the new version, waits until they are ready, and only then stops old pods, so the service keeps answering requests throughout. If the new version turns out to be broken, you can go back to a previous one with a single command.

This lesson explains the two numbers that control the pace, maxSurge and maxUnavailable, shows how the Deployment controller walks from the old state to the new one, and covers the day-to-day commands: watching, pausing, resuming and rolling back. The stand-ins are Python simulations; the real rollouts are driven by the controller from Deployments.

Replace gradually

Surge Start extra new pods first.
Wait New pods must become ready.
Retire Then remove old pods.

Repeat until only the new version remains.

1 / 2

The strategy fields

YAML
spec:
  replicas: 4
  minReadySeconds: 5
  progressDeadlineSeconds: 600
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0
  • maxSurge: the maximum number of pods that may exist above replicas during the update. A percentage is rounded up. Default: 25%.
  • maxUnavailable: the maximum number of pods that may be unavailable (below replicas) during the update. A percentage is rounded down. Default: 25%.
  • They cannot both be zero, because then the controller could neither add nor remove a pod and would be stuck. The API rejects that.
  • minReadySeconds: how long a new pod must stay ready before it counts as available. Use it to catch pods that start, pass one probe, and then crash.
  • progressDeadlineSeconds: if the rollout makes no progress for this long, the Deployment gets the condition Progressing=False (reason ProgressDeadlineExceeded). Kubernetes does not roll back automatically; it only reports it.

Common recipes:

Goal maxSurge maxUnavailable Effect
Zero reduction in capacity 1 (or 25%) 0 Slower; needs spare room in the cluster
Fast, no spare room 0 1 (or 25%) Capacity dips during the update
Fastest 100% 0 Starts a full new set first, doubling resource use briefly

Watching the controller work

The simulation below follows the rules: on each tick pods that were created become ready, then the controller adds new pods up to the surge limit and removes old pods as long as the number of available pods stays at or above replicas - maxUnavailable. The real controller works on ReplicaSet scale operations and reacts to pod events rather than clock ticks, and it does the scaling in a slightly different order, but the limits are the same.

Python Rolling update, tick by tick

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

In the first run (maxSurge: 1, maxUnavailable: 0) the available column stays at 4 for every tick: the controller never lets capacity drop. It takes 8 ticks, because only one extra pod may exist at a time. With the 25% defaults on 4 replicas both limits become 1, the update is faster (5 ticks), but available dips to 3. In the third run maxSurge: 0 means no extra pods, so the controller must remove old pods before it may start new ones, two at a time.

Commands for a rollout

All of these are read-only examples; they need a real cluster.

Terminal
kubectl set image deployment/web web=registry.example.com/web:2.0
kubectl rollout status deployment/web
kubectl rollout pause deployment/web
kubectl rollout resume deployment/web
kubectl rollout undo deployment/web
kubectl rollout history deployment/web

The old --record flag is deprecated; use the kubernetes.io/change-cause annotation shown below instead.

Pause and resume. rollout pause freezes the Deployment: you can make several template edits (image, env, resources) and they will not start a rollout until you run rollout resume, so the changes land together as one revision. A pause while a rollout is in progress stops it where it is, leaving a mix of old and new pods, which is a crude way to run a tiny canary (see deployment strategies for better ones).

Undo. kubectl rollout undo deployment/web goes back to the previous revision; --to-revision=N picks a specific one. Under the hood it copies the old ReplicaSet's template back into the Deployment, which then performs an ordinary rolling update in reverse. Only the pod template is restored; replicas and other fields stay as they are, so a rollback does not undo a scale.

History and change cause. Each distinct pod template is a revision. To make rollout history readable, annotate the Deployment with why you changed it:

Terminal
kubectl annotate deployment/web kubernetes.io/change-cause="upgrade to 2.0"
kubectl rollout history deployment/web
kubectl rollout history deployment/web --revision=3

Revision history and its limit

Old ReplicaSets, scaled to zero, are the history. spec.revisionHistoryLimit (default 10) is how many of them are kept. Once the limit is exceeded, the oldest ones are deleted and you can no longer roll back to them. Setting it to 0 disables rollback entirely.

The stand-in below keeps a history, reuses a ReplicaSet when you roll back or return to an old template, renumbers revisions the way Kubernetes does (the rolled-back-to revision disappears and gets a new number), and prunes old ones.

Python Revision history, rollback and pruning

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

After the undo, revision 3 is gone and the same template reappears as revision 5, marked current. After the next release, revision 1 is pruned, so asking for it fails. This matches what kubectl rollout history shows on a real cluster: revision numbers always increase and never repeat.

When a rollout gets stuck

A rollout that never finishes has a few usual causes, and kubectl rollout status plus kubectl describe deployment will tell you which:

  • The new pods never become ready. Wrong image tag (ImagePullBackOff), a crash on startup (CrashLoopBackOff), or a readiness probe that fails. The old pods stay up, which is the strategy protecting you.
  • No room for the surge. If the cluster has no spare capacity and maxUnavailable is 0, new pods stay Pending and the rollout stalls.
  • A quota limit prevents extra pods from being created.

Fix the cause and re-apply, or run kubectl rollout undo. A rolling update is only as safe as your readiness probes: a probe that reports ready too early lets Kubernetes retire healthy old pods in favor of pods that are not actually serving.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. A Deployment has replicas: 10 and maxSurge: 25%. How many pods may exist at most during the update (counting old and new)?

    25% of 10 is 2.5, and maxSurge rounds up, giving 3 extra pods, so up to 13 in total. maxUnavailable would round down.
  2. 2. Which setting makes the rolling update never reduce serving capacity below the replica count?

    With maxUnavailable 0 the controller must start a new pod and see it available before it removes an old one. Both at 0 is rejected because nothing could move.
  3. 3. A new version crashes on startup and the rollout stalls. What does Kubernetes do by default?

    Kubernetes never rolls back on its own. It stops retiring old pods because the new ones are not ready, and reports ProgressDeadlineExceeded. You (or a tool) must run rollout undo.
  4. 4. What does kubectl rollout undo restore?

    Undo copies the old ReplicaSet's pod template back. Other fields such as the replica count are not changed.
  5. 5. What happens to old revisions when revisionHistoryLimit is exceeded?

    Old ReplicaSets are the history; pruning deletes them, so those revisions are gone.

Summary

  • RollingUpdate replaces pods gradually, bounded by maxSurge (rounded up) and maxUnavailable (rounded down); both cannot be zero.
  • maxUnavailable: 0 with a surge keeps full capacity but needs spare room; maxSurge: 0 needs no extra room but dips capacity.
  • minReadySeconds and good readiness probes are what make the pacing safe.
  • rollout status, pause, resume, undo and history are the working commands; undo restores only the pod template.
  • History is kept as scaled-to-zero ReplicaSets, limited by revisionHistoryLimit; Kubernetes never rolls back by itself.

Next: Deployment strategies.