Skip to content
Tutorials Center
My learning

Topics

Lesson 92 of 111 ยท Orchestration Basics

Why Orchestration?

Understand why running containers at scale needs orchestration: scaling, self-healing, rolling updates and service discovery, with hands-on Docker and Compose limits.

Up to now you have been the orchestrator. You typed docker run, you looked at docker ps, you decided which host a container belongs on, and when something died you noticed and started it again. For a laptop, a side project or a single server running a handful of containers, that is perfectly fine; Docker Compose makes it comfortable.

At some point the job outgrows one person at one terminal. You have ten servers instead of one, thirty containers instead of three, traffic that rises and falls, deployments every day, and a rule that nobody may be woken at 3 a.m. because a process crashed. An orchestrator is software that takes over those chores: you describe the state you want ("three copies of this service, version 1.4, reachable on port 80"), and it works continuously to make reality match.

This lesson explains the problems orchestration solves, and it does so by running into the limits of plain Docker and Compose first, with real commands, so that the features of Swarm and Kubernetes (next lessons) make sense as answers to concrete questions.

The five jobs of an orchestrator

TEXT
 desired state                         what the orchestrator does
 -------------                         --------------------------
 "run 3 copies of web"        ----->   1. SCHEDULING  pick hosts with free CPU/RAM
 "image web:1.4"                       2. SCALING      start/stop copies to match the number
 "restart it if it fails"              3. SELF-HEALING replace failed or unhealthy copies
 "update without downtime"             4. ROLLING UPDATES  replace copies gradually
 "reachable as 'web'"                  5. DISCOVERY & LOAD BALANCING  names and traffic routing

Each one has a manual equivalent in plain Docker. Let us see how far that goes.

1. Self-healing: what Docker does and does not do

Docker has restart policies. They handle the most common failure: the main process of a container exits. We started a container whose process crashes after three seconds, with --restart unless-stopped, and looked twelve seconds later:

CONSOLE
$ docker run -d --name flaky --restart unless-stopped alpine:3 sh -c 'echo started; sleep 3; exit 1'
$ docker inspect -f 'status={{.State.Status}} restarts={{.RestartCount}} exit={{.State.ExitCode}}' flaky
status=restarting restarts=2 exit=1
$ docker logs flaky | sort | uniq -c
      3 started

The daemon restarted the process twice (so it ran three times), waiting a little longer after each failure. That is real self-healing for one host, and it costs nothing. See The Container Lifecycle.

Now two limits. First, a restart policy applies to crashes, not to deliberate stops. If you (or a script) kill the container with docker kill, Docker assumes you meant it:

CONSOLE
$ docker run -d --name victim --restart on-failure nginx:alpine
$ docker kill -s KILL victim
$ docker inspect -f 'status={{.State.Status}} restarts={{.RestartCount}}' victim
status=exited restarts=0

Second, and more important: Docker does not act on failed health checks. A container whose process is alive but broken (deadlocked, out of connections) is marked unhealthy, but it stays exactly where it is:

CONSOLE
$ docker run -d --name stuck --restart unless-stopped \
    --health-cmd 'false' --health-interval 2s --health-retries 2 nginx:alpine
$ docker ps --filter name=stuck --format '{{.Names}} {{.Status}}'
stuck Up 9 seconds (unhealthy)
$ docker inspect -f 'restarts={{.RestartCount}} health={{.State.Health.Status}}' stuck
restarts=0 health=unhealthy

Docker has reported the problem and done nothing about it. An orchestrator treats an unhealthy instance as a failed one: it takes it out of the traffic rotation and starts a replacement. And restart policies cannot help at all when the whole host dies. For that you need a second host and something watching both. (See HEALTHCHECK.)

2. Scaling and service discovery

Scaling by hand means starting more copies and telling the rest of the system about them. Compose can start several replicas of a service, and Docker's embedded DNS gives you service discovery for free. Here is a compose.yaml with a web service (an nginx that answers with its own host name) and a client that sends requests to it:

YAML
services:
  web:
    image: nginx:alpine
    volumes:
      - ./hostname.conf:/etc/nginx/conf.d/default.conf:ro
    restart: unless-stopped
    healthcheck:
      test: ["CMD", "wget", "-qO-", "http://127.0.0.1/"]
      interval: 2s
      retries: 3

  client:
    image: alpine:3
    command: ["sleep", "3600"]
    depends_on:
      web:
        condition: service_healthy

The mounted hostname.conf is a single server block whose only route is return 200 "answered by $hostname\n";. Start three replicas:

CONSOLE
$ docker compose up -d --scale web=3
$ docker compose ps --format 'table {{.Service}}\t{{.Name}}\t{{.Status}}'
SERVICE   NAME                    STATUS
client    orch-client-1           Up 1 second
web       orch-web-1              Up 5 seconds (healthy)
web       orch-web-2              Up 5 seconds (healthy)
web       orch-web-3              Up 4 seconds (healthy)

The name web now resolves to all three container addresses:

CONSOLE
$ docker compose exec client nslookup web | grep Address | tail -3
Address: 172.21.0.2
Address: 172.21.0.4
Address: 172.21.0.3

and requests sent to http://web/ are spread across them. We sent six from the client container and counted the answers:

CONSOLE
$ for i in 1 2 3 4 5 6; do docker compose exec -T client wget -qO- http://web/; done | sort | uniq -c
      3 answered by 09b0fd8fb3ad
      2 answered by c709dd18cbc2
      1 answered by ddcae9645e7b

This is DNS round-robin load balancing: crude (the split is uneven, clients may cache an address, there is no health awareness) but it works with zero setup. It also only works for traffic between containers on the same host's network. It does nothing for outside users: for that you need a proxy (see Nginx and Reverse Proxies) that knows the replicas, and the proxy then has the "names are resolved once" problem we saw there.

What happens when a replica goes away? We stopped one container and asked six more times:

CONSOLE
$ docker stop orch-web-2
$ docker compose ps --format 'table {{.Service}}\t{{.Name}}\t{{.Status}}'
SERVICE   NAME                    STATUS
client    orch-client-1           Up 23 seconds
web       orch-web-1              Up 27 seconds (healthy)
web       orch-web-3              Up 27 seconds (healthy)
$ for i in 1 2 3 4 5 6; do docker compose exec -T client wget -qO- http://web/; done | sort | uniq -c
      5 answered by 09b0fd8fb3ad
      1 answered by ddcae9645e7b

Docker's DNS stopped returning the dead replica, so traffic carried on. But nothing replaced it: the service stayed at two replicas until we ran the scale command again, which noticed the gap and started a new web-2:

CONSOLE
$ docker compose up -d --scale web=3
 Container orch-web-1 Running
 Container orch-web-3 Running
 Container orch-web-2 Starting
 Container orch-web-2 Started

Compose only reconciles when you run it. An orchestrator runs that comparison of "wanted versus actual" in a loop, forever, with no human in it. That loop is called a control loop or reconciliation loop, and it is the central idea of Swarm and Kubernetes.

3. Rolling updates: where Compose stops

Now deploy a new version. We changed the image tag in the Compose file from nginx:alpine to nginx:1.27-alpine and ran docker compose up -d --scale web=3 again:

CONSOLE
$ docker compose up -d --scale web=3 2>&1 | grep -vE "Waiting|Running"
 Container orch-web-1 Recreate
 Container orch-web-2 Recreate
 Container orch-web-3 Recreate
 Container orch-web-2 Recreated
 Container orch-web-3 Recreated
 Container orch-web-1 Recreated
 Container orch-web-3 Starting
 Container orch-web-3 Started
 Container orch-web-1 Starting
 Container orch-web-1 Started
 Container orch-web-2 Starting
 Container orch-web-2 Started
 Container orch-web-3 Healthy
 Container orch-web-1 Healthy
 Container orch-web-2 Healthy
$ docker compose ps --format 'table {{.Name}}\t{{.Image}}\t{{.Status}}'
NAME            IMAGE               STATUS
orch-client-1   alpine:3            Up About a minute
orch-web-1      nginx:1.27-alpine   Up 5 seconds (healthy)
orch-web-2      nginx:1.27-alpine   Up 4 seconds (healthy)
orch-web-3      nginx:1.27-alpine   Up 7 seconds (healthy)

It worked, but look at the order of events: all three replicas were stopped and recreated together, then started. For a few moments there was no healthy web at all. A real rolling update replaces one replica (or a small batch) at a time, waits until the new one is healthy, and only then retires the next old one; and if the new version fails its health check, it stops and rolls back. Compose cannot do that on its own on a single host. Scripts can approximate it (start new containers, check health, switch the proxy, stop old ones) and that is exactly the kind of fragile script an orchestrator replaces. (Editing a bind-mounted file does not even trigger this: Compose compares the container configuration, not the file contents, and left the containers alone when we changed the mounted config. You have to restart or reload explicitly.)

4. Scheduling: which machine?

With one host the question never comes up. With many, somebody must decide where each container runs: which machine has enough free memory and CPU, which already has the image, which must not hold two replicas of the same service (so one failure cannot take both), which has the GPU or the fast disk. That job is scheduling. You express requirements ("I need 512 MB and 0.5 CPU", "spread the replicas across zones") and the scheduler picks. The same resource flags you learned in Resource Limits become the inputs.

Scheduling also means moving containers when a host disappears: the orchestrator notices the node is gone and starts its containers elsewhere. That only works when your containers are stateless or their data is on storage that follows them (see Volume Drivers and Remote Storage); a database with a volume on one disk cannot simply hop to another server.

5. Configuration, secrets and networking across hosts

Two more things break when you leave a single host:

  • Networking. The bridge network from User-Defined Bridge Networks exists on one machine. Containers on different hosts need an overlay network (see Host, None, Macvlan and Overlay) or the orchestrator's own networking.
  • Configuration and secrets. Mounting a .env file on one server is easy; distributing secrets safely to whichever of twenty hosts runs a container is not. Orchestrators store configuration and secrets in the cluster and deliver them to the right containers only.

The landscape

You can sort the options by how much machinery they bring:

Option What it is Fits when
Compose on one host Declarative files, no cluster Dev, CI, small production on a single server with backups
Docker Swarm Built into Docker; a few commands add clustering, services, rolling updates A few hosts, small team, you want Compose-like simplicity (see Docker Swarm Basics)
Kubernetes The dominant orchestrator; large ecosystem, many managed offerings Many services and teams, need for autoscaling, custom networking, a big ecosystem (see From Docker to Kubernetes)
Nomad HashiCorp's scheduler for containers and other workloads You like a simpler scheduler and also run non-container jobs
Managed container services A cloud runs your containers: AWS ECS/Fargate, Google Cloud Run, Azure Container Apps You want orchestration without operating a cluster

(The table is a read-only overview from general knowledge; we did not run these here. Vendors and features change quickly, so check current documentation.) A decision guide for choosing is in Choosing Your Deployment Path.

You may not need an orchestrator. One well-run server with Compose, restart policies, health checks, off-host backups and a reverse proxy serves a surprising amount of traffic. Orchestrators add real operational cost: more concepts, more moving parts, more failure modes. Adopt one when a concrete need appears (more than one host, zero-downtime deployments, autoscaling, many teams), not because it is fashionable. Learn the single-host version well first (see Deploying on a Single Host).

The control loop idea

Whatever product you choose, they share the same design:

TEXT
        +------------------- desired state (files you wrote) ------------------+
        |                                                                      |
        v                                                                      |
   compare  <---------------  observed state (what is really running) ---------+
        |
        +--> differences? ----> start / stop / move / replace containers
        |
        +--> no differences --> do nothing, look again in a moment

You change the desired state by editing a file and submitting it. You never tell the system "start this container on that machine"; you say what must be true and it takes care of the how. This is declarative configuration, the same style as your Compose file, and why those skills carry over. Because the loop never stops, a crashed container, a failed machine or a human mistake is simply a difference that gets corrected.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. A container's health check fails repeatedly. With plain Docker and a restart policy of unless-stopped, what happens?

    In our test the container showed Up (unhealthy) with RestartCount 0. Acting on an unhealthy instance is a job for an orchestrator, which replaces it and removes it from traffic.
  2. 2. You run docker kill on a container that has --restart on-failure. Why is it not restarted?

    We saw status exited with restarts 0 after docker kill. Restart policies are meant for crashes, not for deliberate stops.
  3. 3. What was wrong with the update we ran by changing the image tag and repeating docker compose up -d --scale web=3?

    The output showed Recreate for all three replicas before they were started again. A rolling update replaces them gradually and waits for health between steps.
  4. 4. What is the core idea shared by Swarm and Kubernetes?

    You declare what should be true; the orchestrator reconciles reality with it forever. Compose does the comparison only when you run it.
  5. 5. Which situation most clearly calls for an orchestrator rather than Compose on one server?

    Multiple hosts, failure tolerance and zero-downtime updates are the problems orchestrators solve. The other cases are served well by Compose or even plain docker run.

Summary

  • An orchestrator keeps a cluster in the state you declare: it schedules containers onto hosts, scales them, replaces failed ones, rolls out updates gradually and provides discovery and load balancing.
  • Plain Docker covers part of this on one host: restart policies recover crashed processes, and Compose gives replicas with DNS round-robin.
  • Docker does not replace unhealthy containers, does not restart ones you killed, and cannot help when the host dies.
  • docker compose up reconciles only when you run it, and an image change recreates all replicas together rather than rolling.
  • Scheduling, cross-host networking and distributing secrets and configuration are the other jobs that appear once you have more than one machine.
  • The landscape ranges from Compose on one host, through Swarm and Nomad, to Kubernetes and managed container services. Start simple and adopt an orchestrator when a real need appears.

Next: Docker Swarm Basics