Skip to content
Tutorials Center
My learning

Topics

Lesson 36 of 111 ยท Containers in Depth

Signals, PID 1 and --init

Learn how docker stop really works: SIGTERM and SIGKILL, why PID 1 ignores signals, zombie processes, tini and --init, STOPSIGNAL and stop timeouts.

Every time you run docker stop, Docker does not "turn the container off". It sends a message to one process and hopes that process listens. If it does, your application finishes its requests, flushes its buffers and exits cleanly. If it does not, Docker waits out a grace period and then kills the process without warning. Whether your containers shut down in a fraction of a second or always take ten seconds, and whether a deploy drops in-flight requests, comes down to a handful of facts about Linux signals and process number 1.

In this lesson you will see exactly what Docker sends, why the first process in a container behaves differently from every other process, how to write (or avoid writing) signal handling, what zombie processes are, and how a tiny init program (docker run --init, tini) solves two problems at once. Every session below was run against a real daemon with small Python and shell programs, so the outputs are real. If you have not yet read CMD and ENTRYPOINT, skim its section on exec form first; this lesson builds on it.

What docker stop actually does

A signal is a small numbered message the kernel delivers to a process. A process can install a handler for most signals, which is a function that runs when the signal arrives. The ones you meet in Docker are:

Signal Number Meaning Can the process catch it?
SIGTERM 15 "Please terminate." The polite request. Yes
SIGINT 2 "Interrupt." What Ctrl+C sends. Yes
SIGQUIT 3 "Quit." Some servers use it for a graceful stop. Yes
SIGHUP 1 "Hang up." Many daemons reload config on it. Yes
SIGKILL 9 "Die now." No

docker stop is a two-step process:

TEXT
docker stop web
      |
      |  1. send the stop signal (SIGTERM by default) to PID 1 of the container
      v
  +-----------+   exits in time?   +----------------------------+
  | container | -----------------> | done: exit code is whatever |
  |  PID 1    |                    | the program chose (often 0) |
  +-----------+                    +----------------------------+
      |  no, still running after the grace period
      |  2. send SIGKILL (cannot be caught)
      v
  killed by the kernel: exit code 137 (128 + 9)

Two details matter. First, the signal goes to PID 1 only. Docker does not walk the process tree. Second, the grace period is 10 seconds unless you change it with docker stop -t <seconds>, docker run --stop-timeout <seconds> or stop_grace_period in Compose.

The exit code tells you which path was taken. A process killed by signal N ends with code 128 + N, so 137 is SIGKILL and 143 is SIGTERM. That is why "exit code 137 after docker stop" is the classic sign that your application never reacted to the polite request (see Running Containers for the table of codes).

PID 1 is special

Inside a container, the first process gets process ID 1 in the container's own PID namespace. On any Linux system PID 1 is the init process, and the kernel gives it two special jobs and one special protection.

  1. Signal protection. For every other process, a signal with no handler triggers a default action, and for SIGTERM that default is "terminate". For PID 1 the kernel skips the default action: if PID 1 has not installed a handler, the signal is simply ignored. This stops a stray signal from taking down a whole machine, but it surprises people who run an ordinary program as PID 1.
  2. Adopting orphans. When a process dies, its surviving children are re-parented to PID 1.
  3. Reaping. When a child exits it becomes a "zombie" until its parent reads its exit status. PID 1 must read the status of the orphans it adopted.

Your application was written to be one process among many. It almost certainly installs no handlers for SIGTERM and does not reap strangers' children. Run it as PID 1 and both of those jobs go undone. The next two sections show each symptom.

Symptom 1: a program with no handler ignores docker stop

Here is the smallest possible "deaf" program, and a polite one that installs handlers:

Python
# deaf.py
import time

print("deaf: started, no signal handlers", flush=True)
while True:
    time.sleep(1)
Python
# polite.py
import signal
import sys
import time

def shut_down(signum, frame):
    print(f"polite: got {signal.Signals(signum).name}, cleaning up", flush=True)
    time.sleep(0.5)
    print("polite: bye", flush=True)
    sys.exit(0)

signal.signal(signal.SIGTERM, shut_down)
signal.signal(signal.SIGINT, shut_down)
print("polite: started, waiting for a signal", flush=True)
while True:
    time.sleep(1)

Both are copied into an image built FROM python:3-alpine. First the deaf one. We shorten the grace period with -t 3 so we do not wait the full ten seconds:

CONSOLE
$ docker run -d --name deaf sigdemo python deaf.py
$ time docker stop -t 3 deaf
deaf
real    0m4.0s
$ docker inspect -f 'exit={{.State.ExitCode}}' deaf
exit=137

The stop took the whole grace period (plus a moment of overhead) and ended with 137: Docker sent SIGTERM, Python as PID 1 had no handler so the kernel ignored it, and after three seconds SIGKILL ended it. Now the polite one:

CONSOLE
$ docker run -d --name polite sigdemo
$ time docker stop -t 3 polite
polite
real    0m1.0s
$ docker logs polite
polite: started, waiting for a signal
polite: got SIGTERM, cleaning up
polite: bye
$ docker inspect -f 'exit={{.State.ExitCode}}' polite
exit=0

It returned in about a second (the half second of "cleanup" plus Docker's overhead), ran its handler, and exited with 0. This is what a healthy container looks like.

Most mainstream servers (nginx, PostgreSQL, Redis, Node.js frameworks that call server.close() on SIGTERM, gunicorn, and so on) already handle SIGTERM or SIGQUIT. Plain scripts, one-off tools and some language runtimes do not. Node.js, for example, installs no SIGTERM handler by default, so a bare node server.js as PID 1 ignores docker stop unless your code listens for the signal.

Symptom 2: signals that never arrive

The second trap is a wrapper. If PID 1 is a shell script that starts your real program as a child, Docker's signal goes to the shell, and the shell does not pass it on. The shell is PID 1; your program is just its child:

Terminal
#!/bin/sh
# entry-bad.sh
echo "entrypoint: preparing"
python /app/polite.py
CONSOLE
$ docker run -d --name e sigdemo ./entry-bad.sh
$ docker exec e ps -o pid,ppid,args
PID   PPID  COMMAND
    1     0 {entry-bad.sh} /bin/sh ./entry-bad.sh
    8     1 python /app/polite.py
    9     0 ps -o pid,ppid,args
$ docker stop -t 3 e
e
$ docker logs e
entrypoint: preparing
polite: started, waiting for a signal
$ docker inspect -f 'exit={{.State.ExitCode}}' e
exit=137

Our polite program has a perfectly good handler, but it never ran. The shell received SIGTERM and ignored it as PID 1, the stop waited out the full grace period, and SIGKILL ended everything. Notice that polite: got SIGTERM is missing from the log.

The cure is a single word. exec replaces the shell with the program, so the program becomes PID 1:

Terminal
#!/bin/sh
# entry-good.sh
echo "entrypoint: preparing"
exec python /app/polite.py
CONSOLE
$ docker run -d --name e sigdemo ./entry-good.sh
$ docker exec e ps -o pid,ppid,args
PID   PPID  COMMAND
    1     0 python /app/polite.py
    6     0 ps -o pid,ppid,args
$ docker stop -t 3 e
e
$ docker logs e
entrypoint: preparing
polite: started, waiting for a signal
polite: got SIGTERM, cleaning up
polite: bye
$ docker inspect -f 'exit={{.State.ExitCode}}' e
exit=0

The same rule applies to the Dockerfile. Exec form (CMD ["python", "polite.py"]) starts your program directly. Shell form (CMD python polite.py) wraps it in /bin/sh -c, which, depending on the shell, may or may not exec the last command for you. Do not depend on that luck; write exec form.

A shell can be a fine PID 1 if it traps

If your container really is a shell script that supervises something, give it a trap. The following worker script handles SIGTERM itself:

Terminal
#!/bin/sh
# worker.sh
trap 'echo "worker: SIGTERM received, finishing current job"; sleep 1; echo "worker: done"; exit 0' TERM
echo "worker: ready (pid $$)"
while true; do
  sleep 1 &
  wait $!
done
CONSOLE
$ docker run -d --name w sigdemo ./worker.sh
$ docker stop -t 5 w
w
$ docker logs w
worker: ready (pid 1)
worker: SIGTERM received, finishing current job
worker: done

There is a subtle trick: the loop sleeps in the background and then runs the shell builtin wait. A shell only runs a trap after the current foreground command finishes, so a plain sleep 300 would delay the handler. wait is interrupted immediately by a trapped signal.

Zombies and orphans

The second PID 1 duty is cleaning up. A normal process that exits is not completely gone until its parent calls wait() to collect its exit status. In between, it sits in the process table as a zombie (state Z). Zombies use almost no memory but each holds a process-table slot, and a long-running container that creates thousands of them can eventually fail to start new processes.

When a parent dies first, its children become orphans and the kernel hands them to PID 1. If PID 1 is your application and it never calls wait() for processes it did not start itself, each orphan that finishes stays a zombie forever.

This script reproduces it. Each job starts a shell that launches a background task and exits at once, so the task is orphaned and re-parented to PID 1:

Python
# spawner.py
import subprocess
import time

for n in range(3):
    subprocess.run(["sh", "-c", f"(sleep 1; echo job {n}) > /dev/null & exit 0"])

print("spawner: jobs launched, now idling", flush=True)
while True:
    time.sleep(1)
CONSOLE
$ docker run -d --name z sigdemo python spawner.py
$ sleep 3; docker exec z ps -o pid,ppid,stat,args
PID   PPID  STAT COMMAND
    1     0 S    python spawner.py
    7     1 Z    [sh]
   10     1 Z    [sh]
   12     1 Z    [sh]
   15     0 R    ps -o pid,ppid,stat,args

Three zombies (Z), all children of PID 1, which is a Python script that knows nothing about them. They will stay until the container stops.

The fix: a tiny init process

Real operating systems solve both problems with a dedicated init program that does nothing but forward signals and reap zombies. Containers can do the same. Docker ships one: add --init and it injects a small init binary (docker-init, built on a project called tini) as PID 1, with your command running as its child.

CONSOLE
$ docker run -d --init --name z sigdemo python spawner.py
$ sleep 3; docker exec z ps -o pid,ppid,stat,args
PID   PPID  STAT COMMAND
    1     0 S    /sbin/docker-init -- python spawner.py
    7     1 S    python spawner.py
   17     0 R    ps -o pid,ppid,stat,args

No zombies: the orphaned shells were adopted by docker-init, which reaped them as soon as they finished. And the signal problem disappears too, because docker-init forwards the signals it receives to your program, and your program, no longer PID 1, gets the kernel's ordinary default action. Even the deaf program from before now stops quickly:

CONSOLE
$ docker run -d --init --name deaf sigdemo python deaf.py
$ docker stop -t 3 deaf
deaf
$ docker inspect -f 'exit={{.State.ExitCode}}' deaf
exit=143

It exited almost immediately, with code 143: killed by SIGTERM (128 + 15). The program did not clean up (it has no handler, so the default action simply terminates it), but the stop was fast and the exit status is honest. If you need real cleanup, add a handler like polite.py does. --init gets the signal to your program; it cannot make your program handle it.

In Compose, the same switch is init: true on a service. It lives next to stop_grace_period and stop_signal; see Services in Compose.

Baking tini into the image

--init is a flag on docker run, so everyone who starts your image must remember it. You can bake an init into the image instead. On Alpine the tini package is available:

DOCKERFILE
FROM python:3-alpine
RUN apk add --no-cache tini
WORKDIR /app
COPY spawner.py deaf.py ./
ENTRYPOINT ["/sbin/tini", "--"]
CMD ["python", "spawner.py"]
CONSOLE
$ docker run -d --name t tinydemo
$ sleep 3; docker exec t ps -o pid,ppid,stat,args
PID   PPID  STAT COMMAND
    1     0 S    /sbin/tini -- python spawner.py
    6     1 S    python spawner.py
   16     0 R    ps -o pid,ppid,stat,args
$ docker run --rm tinydemo tini --version
tini version 0.19.0

The -- ends tini's own options so that everything after it is the command to run. On Debian-based images install the tini package with apt-get; on others you can download the static binary from the project's releases and COPY it in. Either way, keep tini as the ENTRYPOINT and your normal command as CMD, so overriding the command still works.

Do you always need an init?

No. You need one when your container (a) runs a program that does not handle SIGTERM, (b) uses a shell wrapper you cannot easily fix, or (c) spawns child processes that may be orphaned (build tools, test runners, process managers, anything that shells out). A single, signal-aware server such as nginx, Postgres or a Go binary that handles SIGTERM and spawns nothing is fine without it. Adding --init anyway costs a few kilobytes and one extra process, so many teams switch it on by default for application containers.

Choosing the stop signal

Not every program shuts down gracefully on SIGTERM. nginx, for instance, treats SIGQUIT as "finish current requests, then exit" and SIGTERM as "fast shutdown". The official nginx image says so in its metadata:

CONSOLE
$ docker run -d --name n nginx:alpine
$ docker inspect -f '{{.Config.StopSignal}}' n
SIGQUIT

That value comes from the image's STOPSIGNAL instruction (see WORKDIR, USER, EXPOSE and LABEL). You can set it at run time as well:

CONSOLE
$ docker run -d --name k --stop-signal SIGINT sigdemo
$ docker inspect -f '{{.Config.StopSignal}}' k
SIGINT
$ docker stop k
k
$ docker logs k
polite: started, waiting for a signal
polite: got SIGINT, cleaning up
polite: bye

To send any other signal without stopping, use docker kill -s. Many daemons reload their configuration on SIGHUP, but our program does not handle it, and because it is PID 1 the signal is silently ignored:

CONSOLE
$ docker run -d --name k sigdemo
$ docker kill -s HUP k
k
$ docker ps --filter name=k --format '{{.Names}} {{.Status}}'
k Up 5 seconds
$ docker kill -s INT k
k
$ docker logs k
polite: started, waiting for a signal
polite: got SIGINT, cleaning up
polite: bye

docker kill with no -s sends SIGKILL; with -s it only delivers a signal and the container continues running if the program survives it.

Tuning the grace period

Ten seconds is a guess. A web server with long uploads may need a minute to drain; a stateless worker may need two seconds. Set it where it belongs:

CONSOLE
$ docker run -d --name g --stop-timeout 2 sigdemo python deaf.py
$ docker inspect -f '{{.Config.StopTimeout}}' g
2
$ time docker stop g
g
real    0m3.0s
$ docker inspect -f 'exit={{.State.ExitCode}}' g
exit=137

The container's own timeout (2) was used because we gave docker stop no -t. An explicit docker stop -t overrides it for that one call. In Compose, the same settings are one block:

YAML
name: demo
services:
  app:
    build: .
    init: true
    stop_grace_period: 20s
    stop_signal: SIGINT
CONSOLE
$ docker compose up -d --build
$ docker inspect -f 'init={{.HostConfig.Init}} signal={{.Config.StopSignal}} timeout={{.Config.StopTimeout}}' demo-app-1
init=true signal=SIGINT timeout=20
$ docker compose exec app ps -o pid,args
PID   COMMAND
    1 /sbin/docker-init -- python polite.py
    6 python polite.py
    7 ps -o pid,args

Raising the grace period is not a substitute for handling the signal. If your program ignores SIGTERM, a longer grace period just makes every deploy slower.

Writing a good shutdown handler

When you control the application, aim for this sequence on SIGTERM:

  1. Stop accepting new work (close the listening socket, stop pulling jobs from the queue).
  2. Finish what is already in flight, within a bounded time shorter than the grace period.
  3. Release resources: flush logs, close database connections, commit offsets.
  4. Exit with status 0.

Keep the handler short and set a flag or call your framework's shutdown API rather than doing heavy work inside the signal callback. Most web frameworks have a built-in "graceful shutdown" hook; use it.

Checklist and pitfalls

  • Run docker exec <container> ps -o pid,ppid,args and look at PID 1. Is it your program, a shell, or an init? That single command diagnoses most shutdown trouble.
  • A shell script entrypoint must end with exec "$@" (or exec your-program). See the pattern in CMD and ENTRYPOINT.
  • Use exec form for CMD and ENTRYPOINT.
  • Exit code 137 on stop means SIGKILL; 143 means SIGTERM ended the process; 0 means a clean exit.
  • docker run -it forwards Ctrl+C as SIGINT to PID 1. If your program has no SIGINT handler and is PID 1, Ctrl+C appears to do nothing. --init fixes that too.
  • Signals reach PID 1 only. Process managers inside containers (supervisord and similar) must forward them to their children themselves; a container with several services is usually a design smell (see Container Lifecycle).
  • Long-running containers that shell out heavily should use an init even if their main program handles SIGTERM, to avoid slow zombie accumulation.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Which process receives the signal when you run docker stop?

    Docker sends the stop signal (SIGTERM unless the image or run options change it) to PID 1 only. If PID 1 is a shell that does not forward it, your application never hears about it.
  2. 2. A container takes the full grace period to stop and docker inspect shows exit code 137. What most likely happened?

    137 = 128 + 9, the exit status of a process killed by SIGKILL. That happens when the stop signal was ignored or never delivered, and the grace period ran out.
  3. 3. What does exec do at the end of an entrypoint shell script such as exec python app.py?

    exec replaces the shell process with the new program, keeping the same PID. Without it, the shell stays as PID 1 and the program is a child that never sees SIGTERM.
  4. 4. What two problems does docker run --init address?

    The injected init (docker-init, based on tini) forwards signals to your command and reaps orphaned children. It cannot add cleanup logic to your program; for that you still need a signal handler.
  5. 5. Where can you set a different grace period for one service in Compose?

    stop_grace_period sets how long Compose waits after the stop signal before sending SIGKILL. docker stop -t and docker run --stop-timeout do the same outside Compose.

Summary

  • docker stop sends one signal (SIGTERM by default) to PID 1, waits the grace period (10 s by default), then sends SIGKILL.
  • PID 1 has no default signal actions: a program with no handler running as PID 1 ignores SIGTERM, so it is killed after the timeout with exit code 137.
  • A shell wrapper as PID 1 does not forward signals. End scripts with exec, and use exec form in the Dockerfile.
  • PID 1 must also reap orphaned children; an application that does not will pile up zombie (Z) processes.
  • docker run --init (Compose: init: true) or tini in the image forwards signals and reaps zombies. Your program still needs its own handler for real cleanup.
  • Change the stop signal with STOPSIGNAL, --stop-signal or stop_signal, and the grace period with -t, --stop-timeout or stop_grace_period.
  • Check docker exec <name> ps -o pid,ppid,args to see who PID 1 really is.

Next: Users and Permissions