Skip to content
Tutorials Center
My learning

Topics

Lesson 23 of 38 ยท Secrets and Configs

Protecting Swarm Secrets: Autolock and Hygiene

Understand how Swarm encrypts Raft data, what autolock and the unlock key do, how to rotate keys, and a checklist for keeping secrets out of logs, images and Git.

Swarm secrets are only as safe as the managers that store them. Every manager keeps the full Raft log on disk, including all secrets. This lesson explains how that data is encrypted, what autolock adds, and the habits that keep secrets out of places they should never be.

As elsewhere in this track the swarm itself cannot run on this site, so commands are read-only text. The runnable blocks are labelled stand-ins that model key handling and a small secret scanner.

Two keys protect the Raft log

When a swarm is created, each manager generates two keys and stores them on its disk:

  1. A TLS key that identifies the node to the rest of the swarm.
  2. A data encryption key (DEK) that encrypts the Raft log (which contains secrets and configs).

By default both keys sit on disk next to the data, so someone who copies the whole /var/lib/docker/swarm directory from a stolen disk or backup can read your secrets. Encryption at rest protects against casual loss of storage, but not against a stolen full copy.

Autolock

Autolock closes that gap. With autolock on, Docker encrypts the two keys themselves with a separate unlock key that is never stored on the manager's disk. After a manager restarts, it stays locked: it cannot read the Raft log, cannot join the swarm and cannot answer commands until an operator supplies the unlock key.

Terminal
docker swarm init --autolock
docker swarm update --autolock=true
docker swarm unlock-key

docker swarm init --autolock prints the unlock key once, as a string beginning with SWMKEY-1-. Store it in a password manager or vault, separately from the manager hosts and from backups of /var/lib/docker/swarm.

After a restart of a locked manager:

Terminal
docker swarm unlock

Docker prompts for the key. Until then docker node ls on that node fails with a message that the swarm is encrypted and needs unlocking.

Trade-off: autolock improves security and costs convenience. If every manager reboots (for example after a data-center power cut) and nobody has the key, the swarm is stuck. That is why the key must live in a place your on-call people can reach.

Rotating keys

Terminal
docker swarm unlock-key --rotate
docker swarm ca --rotate

--rotate generates a new unlock key and invalidates the old one, which is the right response if the key may have leaked. docker swarm ca --rotate replaces the swarm's certificate authority and reissues node certificates without downtime. Rotating the CA is also how you move to an external CA. Schedule both periodically and keep the new key in your vault before discarding the old one.

Node certificates are renewed automatically every 90 days by default (docker swarm update --cert-expiry 2160h). You normally do not touch them.

flowchart TD
  U["Unlock key<br/>(in your vault)"] -->|encrypts| K["TLS key + DEK<br/>(on manager disk)"]
  K -->|decrypts| R["Raft log<br/>(secrets, configs)"]

With autolock the chain starts at a key that is not on the machine.

Secret hygiene checklist

  • Create secrets from files or stdin, never as command arguments (they end up in shell history and ps).
  • Do not commit secret files to Git. Add them to .gitignore and use a secrets manager as the source of truth.
  • Do not print secrets from the application, including in debug logs and error pages.
  • Do not mount a secret into a service that does not need it.
  • Limit who can reach the Docker socket on a manager: whoever controls the API can create a service that mounts any secret and prints it. Treat manager access as root on the whole cluster.
  • Run managers on dedicated nodes (no application workloads) so a compromised app cannot reach the Docker socket.
  • Rotate on a schedule and after staff changes.
  • Prefer short-lived credentials (for example database users created per rotation) so a leaked one expires.

Stand-in: simulating the lock

Python Stand-in: why autolock needs the key

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Python Stand-in: scan files for secret-looking lines

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Common mistakes

  • Enabling autolock and losing the unlock key. Test the recovery procedure while nothing is on fire.
  • Backing up /var/lib/docker/swarm and the unlock key to the same place.
  • Giving developers access to a manager node "just to look around".
  • Assuming Raft encryption makes docker config safe for passwords; it does not change what inspect shows.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does autolock add to default Swarm encryption at rest?

    Without autolock the keys sit on disk next to the Raft data. Autolock protects them with a separate unlock key.
  2. 2. A manager reboots in a swarm with autolock enabled. What must happen before it works again?

    The locked manager cannot read its Raft log until the operator provides the key.
  3. 3. Which command invalidates a possibly leaked unlock key?

    Rotating generates a new key and invalidates the old one.
  4. 4. Why is access to a manager's Docker socket equivalent to access to all secrets?

    API access lets a user grant secrets to a service they control.

Summary

  • The Raft log (with secrets and configs) is encrypted at rest, but by default the keys are on the same disk.
  • Autolock protects those keys with an unlock key you keep elsewhere; the cost is a manual unlock after restarts.
  • Rotate the unlock key and the CA periodically, and immediately after a suspected leak.
  • Keep secrets out of Git, logs, images and arguments, and treat manager access as root.

Next: Compose Files for Swarm Stacks