Skip to content
Tutorials Center
My learning

Topics

Lesson 6 of 38 ยท Swarm Architecture

Managers and Workers

Understand the two node roles in Docker Swarm, what managers do, the leader, how workers behave, and how to size and place manager nodes.

Every node in a swarm has one of two roles: manager or worker. Understanding the difference explains most of the cluster's behaviour, from "why can't I run docker service ls on this machine?" to "why did the whole cluster freeze when two machines died?".

What a manager does

Managers are the brain. Each manager:

  • accepts API calls (the Docker CLI talks to the Engine on a manager to create services),
  • keeps a replica of the cluster state in a built-in database,
  • takes part in leader election,
  • runs the orchestrator, allocator, scheduler and dispatcher components that turn services into tasks and assign them,
  • acts as the cluster's certificate authority.

State, which includes nodes, services, tasks, secrets, configs and networks, is stored in a Raft log kept on every manager (see Raft and quorum). The data lives on disk under /var/lib/docker/swarm on each manager.

Among the managers one is the leader. Only the leader makes changes; the others forward requests to it and replicate its log. If the leader fails, the remaining managers elect a new one within seconds.

What a worker does

Workers are the muscle. A worker:

  • runs the tasks that managers assign,
  • reports task status back,
  • does not hold cluster state or secrets it was not given,
  • cannot run swarm management commands (docker service ..., docker node ... fail with a message that the node is not a manager).

Workers are cheap to add and remove, so most clusters have few managers and many workers.

Managers can run tasks too

By default managers are also schedulable. A three-node swarm where all three are managers will run your containers on all three. For small clusters that is practical. For larger ones it is better to keep managers free of application load so a runaway container cannot starve the control plane. You do that by draining them (see Node management):

CONSOLE
$ docker node update --availability drain manager1

Roles at a glance

Manager or worker?

Manager Stores state and schedules
Worker Runs the containers
Leader The one manager that writes

Leader is a manager, not a third role.

1 / 2

How many managers?

More managers does not mean more speed. Every write must be acknowledged by a majority, so extra managers add latency. They add fault tolerance, and only up to a point:

Managers Majority needed Failures tolerated
1 1 0
3 2 1
5 3 2
7 4 3

Docker recommends an odd number and no more than seven. Use three for most clusters, five if you must survive two simultaneous failures. Spread them across failure domains (racks, availability zones) when you can, but keep their network latency low, because the Raft protocol is sensitive to slow links.

Reaching the cluster

The CLI talks to whichever Engine you point it at. To administer, connect to any manager (via SSH, or with a Docker context):

CONSOLE
$ docker context create swarm-prod --docker "host=ssh://[email protected]"
$ docker context use swarm-prod

A request sent to a non-leader manager is forwarded to the leader transparently, so it does not matter which manager you use.

Seeing roles

CONSOLE
$ docker node ls
$ docker node inspect --format '{{.Spec.Role}} {{.ManagerStatus.Leader}}' manager1

The MANAGER STATUS column of docker node ls shows Leader, Reachable (healthy follower) or Unreachable (a manager that the leader cannot contact), and is empty for workers.

Stand-in: the role rule

This stand-in implements the permission rule so you can test which node may run which command.

Python Stand-in: who may run which command?

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Stand-in: spare capacity of the control plane

A common design question: how much application load should a manager carry? This calculator shows the memory left for workloads when you reserve part of each manager for the control plane.

Python Stand-in: capacity planning

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Check your understanding

Quick check ยท 3 questions

of 3 answered You scored / 3 ยท perfect!

  1. 1. Which node type stores the cluster state and takes part in elections?

    Managers keep the Raft log and vote for a leader. Workers do neither.
  2. 2. A worker runs docker service ls. What happens?

    Only managers can query or change cluster state.
  3. 3. How many manager failures can a 5-manager swarm tolerate?

    A majority of 3 is needed, so up to 2 managers can fail.

Summary

  • Managers hold state and schedule; workers run tasks.
  • One manager is the leader; others replicate and forward.
  • Use three or five managers (never more than seven), preferably not running app load in larger clusters.
  • Any manager can accept your commands.

Next: Raft Consensus and Quorum