Skip to content
Tutorials Center
My learning

Topics

Lesson 62 of 111 ยท Docker Compose

Startup Order and Health

Stop 'connection refused' at startup: learn depends_on conditions, healthchecks, service_completed_successfully for migrations, up --wait, and restart policies in Compose.

Sooner or later every Compose project hits the same bug: the app starts, tries to connect to the database, and crashes with "connection refused". The database container was started first, as depends_on promised. It just was not ready yet. A database spends a few seconds initializing before it accepts connections, and Docker has no way of knowing that unless you tell it how to check.

This lesson fixes the problem properly. You will see the failure with your own eyes, add a health check, use depends_on conditions to wait for it, run a one-shot migration before the app, and learn what happens when a dependency never becomes healthy. It builds on HEALTHCHECK and Services in Compose. All output is from Docker 29 and Compose v5.

The race: started is not ready

Here is the broken setup. The app service connects once and exits, which makes the failure easy to see:

YAML
name: tcd-b9-dh
services:
  db:
    image: postgres:17-alpine
    environment:
      POSTGRES_PASSWORD: example
  app:
    image: postgres:17-alpine
    environment:
      PGPASSWORD: example
    command: ["sh", "-c", "psql -h db -U postgres -Atc 'select 1'"]
    depends_on:
      - db
CONSOLE
$ docker compose up
$ docker compose logs app
app-1  | psql: error: connection to server at "db" (172.21.0.2), port 5432 failed: Connection refused
app-1  | 	Is the server running on that host and accepting TCP/IP connections?
$ docker compose ps -a --format 'table {{.Service}}\t{{.Status}}'
SERVICE   STATUS
app       Exited (2) About a minute ago
db        Up About a minute

depends_on: [db] guarantees one thing: Compose started the db container before the app container. It does not wait for Postgres to finish initializing the data directory, so app arrived at a closed door. Applications that retry their connection hide this problem; applications that do not, fail.

TEXT
 time ->
 db:   [container starts][ initdb ......... ][ accepting connections ]
 app:               [container starts][connect] <- too early: refused
                                                   ^ depends_on stops here

Step 1: teach Docker what "ready" means

A health check is a command Docker runs inside the container at intervals. If it exits 0 the container is healthy; if it keeps failing it is unhealthy. For Postgres the right probe is pg_isready:

YAML
services:
  db:
    image: postgres:17-alpine
    environment:
      POSTGRES_PASSWORD: example
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres"]
      interval: 2s
      timeout: 3s
      retries: 10
      start_period: 5s
Key Meaning
test The command. CMD runs an argument list directly; CMD-SHELL runs a string through the shell. Exit 0 means healthy.
interval Time between checks.
timeout A check that takes longer counts as a failure.
retries Consecutive failures before the status becomes unhealthy.
start_period Grace time at startup: failures during it do not count towards retries.

The same options exist in a Dockerfile (see HEALTHCHECK); a Compose healthcheck overrides the image's. Pick a probe that tests real readiness, not just "the process exists". For a web service, wget -q --spider http://localhost:8080/health or curl -f is better than checking the PID; for Redis, redis-cli ping.

Step 2: wait for it with a condition

Now change the dependency to the long form, with a condition:

YAML
  app:
    image: postgres:17-alpine
    depends_on:
      db:
        condition: service_healthy

There are three conditions:

Condition Waits until
service_started The container has started (the default; same as the short list form)
service_healthy The health check reports healthy
service_completed_successfully The container exited with code 0

With service_healthy, Compose creates app only after db reports healthy. Compose prints each wait as it happens:

CONSOLE
$ docker compose up -d
 Container tcd-b9-dh-db-1 Started
 Container tcd-b9-dh-db-1 Waiting
 Container tcd-b9-dh-db-1 Healthy
 Container tcd-b9-dh-app-1 Started
$ docker compose ps -a --format 'table {{.Service}}\t{{.Status}}'
SERVICE   STATUS
app       Up 1 second
db        Up 7 seconds (healthy)

The (healthy) badge in docker compose ps comes from the same check, so you can also see health at a glance.

Step 3: one-shot jobs with service_completed_successfully

Many stacks need something to run once before the app: a database migration, a seed script, a cache warm-up. Model it as its own service that exits, and make the app depend on it finishing with code 0:

YAML
name: tcd-b9-dh
services:
  db:
    image: postgres:17-alpine
    environment:
      POSTGRES_PASSWORD: example
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres"]
      interval: 2s
      timeout: 3s
      retries: 10
      start_period: 5s
  migrate:
    image: postgres:17-alpine
    environment:
      PGPASSWORD: example
    command:
      - sh
      - -c
      - psql -h db -U postgres -c 'CREATE TABLE t(id int)' && echo migrated
    depends_on:
      db:
        condition: service_healthy
  app:
    image: postgres:17-alpine
    environment:
      PGPASSWORD: example
    command:
      - sh
      - -c
      - psql -h db -U postgres -Atc 'select count(*) from t'; sleep 300
    depends_on:
      db:
        condition: service_healthy
      migrate:
        condition: service_completed_successfully

A chain forms: db becomes healthy, then migrate runs, then app starts. The app query only works if the table exists, so it proves the order:

CONSOLE
$ docker compose up -d
 Container tcd-b9-dh-db-1 Started
 Container tcd-b9-dh-db-1 Waiting
 Container tcd-b9-dh-db-1 Healthy
 Container tcd-b9-dh-migrate-1 Started
 Container tcd-b9-dh-migrate-1 Waiting
 Container tcd-b9-dh-migrate-1 Exited
 Container tcd-b9-dh-app-1 Started
$ docker compose ps -a --format 'table {{.Service}}\t{{.Status}}'
SERVICE   STATUS
app       Up 1 second
db        Up 7 seconds (healthy)
migrate   Exited (0) 1 second ago
$ docker compose logs migrate app
migrate-1  | CREATE TABLE
migrate-1  | migrated
app-1      | 0

Exited (0) for migrate is the success state here, not a problem. The 0 printed by app is the row count of the new, empty table.

Compose runs migrate again on the next docker compose up, because it exited and the service is part of the project. Our CREATE TABLE t is not idempotent, so a second run fails, and with service_completed_successfully the app is blocked (output below).

CONSOLE
$ docker compose up -d --wait
 Container tcd-b9-dh-migrate-1 Error service "migrate" didn't complete successfully: exit 1
service "migrate" didn't complete successfully: exit 1

Write migration steps so they can run any number of times (CREATE TABLE IF NOT EXISTS, or a migration tool that tracks what it has applied).

When a dependency never becomes healthy

If a health check keeps failing, up does not hang forever. Here a deliberately sick service uses test: ["CMD", "false"] with interval: 1s and retries: 2:

CONSOLE
$ docker compose up -d
 Container tcd-b9-dh-sick-1 Started
 Container tcd-b9-dh-sick-1 Waiting
 Container tcd-b9-dh-sick-1 Error dependency sick failed to start
dependency failed to start: container tcd-b9-dh-sick-1 is unhealthy
$ docker compose ps -a --format 'table {{.Service}}\t{{.Status}}'
SERVICE   STATUS
after     Created
sick      Up 3 seconds (unhealthy)

The dependent service after was created but never started, and Compose reports the reason. That message, dependency failed to start: container ... is unhealthy, is what you will see in real projects when a database never gets ready (wrong password, full disk, a crash loop). Read the dependency's logs with docker compose logs sick.

up --wait

docker compose up -d returns as soon as containers are started. Add --wait and it returns only when every service is running and, where a health check exists, healthy. This is excellent for scripts and CI:

Terminal
docker compose up -d --wait && ./run-smoke-tests.sh

If a service ends unhealthy or exits with an error, the command fails with a non-zero exit code, so your pipeline stops instead of testing a broken stack. See Docker in CI.

Restart policies and health

Health checks and restart are separate. Docker does not restart a container just because it is unhealthy; the status is information. Restart policies react to the process exiting:

  • restart: unless-stopped brings the container back after a crash or a reboot.
  • restart: on-failure:5 retries a failing process up to five times.
  • A container that hangs but does not exit stays unhealthy until you act (or an orchestrator does; see Why Orchestration?).

For dependencies, the long depends_on form also takes restart: true, which asks Compose to restart the dependent service when the dependency is restarted by a Compose command such as docker compose restart db.

Make the app resilient anyway

Health-based ordering removes most startup races, but a database can still restart while the app runs. The sturdy answer is both: use service_healthy for a smooth start, and make your application retry connections with a short back-off. Good libraries do this already. Treat depends_on as a convenience for cold starts, not as a guarantee for the lifetime of the stack.

Debugging checklist

  1. docker compose ps -a shows state and health for every service, including exited ones.
  2. docker compose logs <service> shows why a health check or program failed.
  3. docker inspect <container> --format '{{json .State.Health}}' shows the last health probe results (see Listing and Inspecting).
  4. docker compose config shows the dependency conditions Compose actually resolved.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. app has depends_on: [db] and still gets 'connection refused'. Why?

    The short form means service_started. Postgres needs seconds after the container starts before it accepts connections, so the app connected too early.
  2. 2. Which condition runs a migration container to completion before the app starts?

    service_completed_successfully waits until the dependency exits with code 0. If it exits non-zero, the dependent service does not start.
  3. 3. What does start_period in a healthcheck do?

    start_period gives slow-starting programs time to boot. Failed probes in that window are not counted as failures.
  4. 4. A container becomes unhealthy. What does Docker do on its own?

    Health is a status, not an action. Restart policies react to exits. An orchestrator can act on health, but plain Docker does not.
  5. 5. Why use docker compose up -d --wait in a CI script?

    --wait blocks until the stack is up (and healthy where a check exists), giving the next CI step a ready stack, or a failing exit code.

Summary

  • depends_on alone only controls start order; it does not wait for readiness.
  • Add a healthcheck that tests real readiness (pg_isready, redis-cli ping, an HTTP /health), then depend on it with condition: service_healthy.
  • Use service_completed_successfully for one-shot jobs such as migrations, and make them idempotent because they run on every up.
  • A dependency that never becomes healthy makes up fail with dependency failed to start.
  • up -d --wait blocks until the stack is healthy, which suits scripts and CI.
  • Docker does not restart unhealthy containers; restart policies react to exits, so keep applications able to retry connections.

Next: Compose for Development