Skip to content
Tutorials Center
My learning

Topics

Lesson 10 of 78 ยท Pods

Pod Lifecycle and Phases

Follow a Pod from Pending to Running to Succeeded or Failed. Learn container states, restartPolicy, graceful termination and how CrashLoopBackOff backoff works.

A Pod is born, runs for a while and then ends, and Kubernetes records where it is in that journey. You see the result every time you run kubectl get pods: Pending, Running, Completed, CrashLoopBackOff. Some of these words are real phases of the Pod, and some are friendly summaries that kubectl builds from more detailed container states. Mixing them up is the number one cause of confusion when debugging.

In this lesson you learn the five Pod phases, the three container states, what restartPolicy decides, how a Pod shuts down gracefully, and why a crashing container waits longer and longer between restarts. Two Python stand-ins let you watch the rules work. You should know what a Pod is first (Pods).

Manifests and kubectl output on this page are read-only examples. The Python programs are simplified models of kubelet behavior, not the kubelet itself.

Pod phases

The Pod's status.phase is a small, coarse summary. There are exactly five values:

Phase Meaning
Pending The Pod was accepted, but one or more containers are not running yet. It may be waiting to be scheduled, or pulling images.
Running The Pod is bound to a node and at least one container is running, or starting, or restarting.
Succeeded All containers ended with exit code 0 and will not be restarted.
Failed All containers have ended and at least one failed (non-zero exit or killed by the system), and will not be restarted.
Unknown The node could not be reached, so the state could not be determined. Rare; it indicates a node problem.

Notice what is not a phase: CrashLoopBackOff, ImagePullBackOff, Terminating, Completed and OOMKilled. Those strings in the STATUS column come from container states and reasons, which kubectl shows in place of the phase when they are more useful. A Pod whose container keeps crashing is, technically, still in the Running phase.

stateDiagram-v2
    [*] --> Pending
    Pending --> Running: scheduled and a container starts
    Pending --> Failed: cannot be started
    Running --> Succeeded: all containers exit 0
    Running --> Failed: a container fails, no restart
    Succeeded --> [*]
    Failed --> [*]

The Pod phases. Restarts of a single container happen inside the Running phase.

Container states

Each container inside the Pod has its own state, shown under status.containerStatuses and in kubectl describe pod:

  • Waiting: the container is not running yet. A reason explains why: ContainerCreating, ImagePullBackOff, ErrImagePull, CrashLoopBackOff, CreateContainerConfigError.
  • Running: the container process started and has not exited.
  • Terminated: the process ended. It records exitCode, a reason such as Completed, Error or OOMKilled, and timestamps.

A useful trick: a restarting container shows its previous run as lastState, so after a crash you can still read the old exit code and reason.

TEXT
State:          Waiting
  Reason:       CrashLoopBackOff
Last State:     Terminated
  Reason:       Error
  Exit Code:    1
Restart Count:  4

That output (read-only example) says the container is currently waiting out a backoff delay, the last run exited with code 1, and it has been restarted four times. Exit code conventions help: 0 is success, 1 is a generic app error, 137 means killed by SIGKILL (128 + 9, often an out-of-memory kill), and 143 means ended by SIGTERM (128 + 15).

restartPolicy

The kubelet restarts containers according to the Pod-level field restartPolicy:

Value Restarts a container when... Typical use
Always (default) it exits for any reason, even with code 0 web servers, workers
OnFailure it exits with a non-zero code batch tasks that should retry
Never never one-shot tasks you want to inspect
YAML
apiVersion: v1
kind: Pod
metadata:
  name: one-shot
spec:
  restartPolicy: Never
  containers:
    - name: task
      image: busybox:1.37
      command: ["sh", "-c", "echo working; exit 3"]

With Never this Pod ends in Failed after the first run. Notice that Deployments only allow Always, because a Deployment's whole purpose is to keep things running. For work that ends, use a Job (Jobs and CronJobs). Restarts always happen on the same node: the kubelet restarts containers, but it does not move the Pod.

CrashLoopBackOff and the backoff delay

If a container keeps exiting, restarting it instantly in a tight loop would waste CPU and flood the logs. The kubelet instead waits between restarts, and the wait doubles each time: 10 seconds, 20, 40, 80, 160, and then it stays at a cap of 5 minutes. The delay resets once the container has run successfully for 10 minutes. While it waits, you see CrashLoopBackOff.

CrashLoopBackOff is therefore not an error cause. It is a symptom that says "this container keeps dying; I am slowing down". The cause is in the logs and the exit code: kubectl logs POD --previous shows the output of the last crashed run.

Here is a stand-in that models the delay. It does not start anything. It only does the arithmetic of the restart schedule, so you can see how fast the waiting grows.

Python The CrashLoopBackOff schedule

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Notice the shape: the first few restarts happen within a minute or two, and after about the sixth the container is tried once every five minutes. That is why a crashing Pod can look "stuck" for a long time after you fix the cause. Deleting the Pod (so a controller makes a new one) skips the wait.

A small state machine

The second stand-in simulates a container under all three restartPolicy values. It follows the real rules: the phase is computed from what happened, OnFailure restarts only on non-zero exit codes, and the backoff delay doubles. The application is described as a list of exit codes: each entry is how one run ends.

Python Pod phase under each restartPolicy

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The Always case is worth a second look. Even though the program exits cleanly with code 0 every time, a Pod with restartPolicy: Always never reaches Succeeded. It restarts forever. That is correct for a server, and a classic bug for a script that was meant to run once.

Graceful termination

When a Pod is deleted, the API server does not kill it at once. It sets a deletion timestamp, and the Pod enters Terminating (shown by kubectl, not a phase). The kubelet then runs this sequence:

  1. The Pod is removed from Service endpoints, so new traffic stops arriving. This happens in parallel with step 2, which is why short delays matter (see below).
  2. If a preStop hook is defined, it runs first.
  3. The container receives SIGTERM (the signal that means "please shut down").
  4. The kubelet waits up to terminationGracePeriodSeconds (default 30).
  5. Any process still alive gets SIGKILL, which cannot be caught.
YAML
spec:
  terminationGracePeriodSeconds: 45
  containers:
    - name: api
      image: example/api:1.4
      lifecycle:
        preStop:
          exec:
            command: ["sh", "-c", "sleep 5"]

Two practical consequences. First, your application must handle SIGTERM: stop accepting new work, finish in-flight requests, then exit. A process that ignores SIGTERM always burns the full grace period and ends with exit code 137. Second, in containers started through a shell script, SIGTERM may reach the shell instead of your program unless the script uses exec. This is the same issue described in Docker's CMD and ENTRYPOINT lesson. The brief sleep in preStop is a common trick that gives load balancers a moment to drop the Pod before the process stops serving.

Reading phases in practice

A few commands cover most lifecycle questions (read-only examples):

Terminal
kubectl get pod web-0 -o jsonpath='{.status.phase}'
kubectl describe pod web-0          # states, reasons, events
kubectl logs web-0 --previous       # output of the last crashed run
kubectl get pod web-0 -w            # watch the status change live

If a Pod is Pending for a long time, the answer is in the events at the bottom of describe: no node had enough resources, an image cannot be pulled, or a volume cannot be attached. Troubleshooting Playbook walks through these cases one by one.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Which of these is a real Pod phase?

    The five phases are Pending, Running, Succeeded, Failed and Unknown. The others are container reasons or kubectl display strings.
  2. 2. A container exits with code 0 and the Pod has restartPolicy: Always. What happens?

    Always restarts the container regardless of exit code, so the Pod never reaches Succeeded. Use a Job or restartPolicy OnFailure/Never for work that should end.
  3. 3. What does CrashLoopBackOff tell you?

    It is a symptom, not a cause. The delay doubles from 10 seconds up to 5 minutes. Look at the logs and exit code to find the real reason.
  4. 4. What happens when the termination grace period ends and a process is still running?

    After SIGTERM and the grace period (30 seconds by default), the kubelet sends SIGKILL, which the process cannot catch or ignore.

Summary

  • A Pod has one of five phases: Pending, Running, Succeeded, Failed or Unknown. Strings such as CrashLoopBackOff come from container states.
  • Containers are Waiting, Running or Terminated, and lastState keeps the previous run's exit code and reason.
  • restartPolicy (Always, OnFailure, Never) decides restarts; the kubelet restarts containers on the same node.
  • Restart delays double from 10 seconds up to a 5-minute cap and reset after 10 minutes of healthy running.
  • Deletion sends SIGTERM, waits terminationGracePeriodSeconds, then SIGKILL. Handle SIGTERM in your app.

Next, learn how Kubernetes decides a container is healthy in Liveness, Readiness and Startup Probes.