Skip to content
Tutorials Center
My learning

Topics

Lesson 31 of 38 ยท Scaling and Updates

Resource Limits and Reservations in Docker Swarm

Control CPU and memory for Swarm services: limits versus reservations, how the scheduler uses them, OOM kills, ulimits, and how to size a service from measurements.

Containers share a node. Without boundaries, one memory-hungry service can starve its neighbours or trigger the Linux out-of-memory (OOM) killer on a whole machine. Swarm offers two different tools for this and people often mix them up: limits (a cap at runtime) and reservations (a promise used by the scheduler).

Swarm is not available on this site, so the YAML and commands are read-only, and the Python blocks are labelled stand-ins that model scheduling and OOM decisions.

Limits and reservations

YAML
services:
  api:
    image: myorg/api:1.5
    deploy:
      replicas: 4
      resources:
        limits:
          cpus: "0.50"
          memory: 512M
        reservations:
          cpus: "0.25"
          memory: 256M

The same on the command line:

Terminal
docker service update \
  --limit-cpu 0.5 --limit-memory 512M \
  --reserve-cpu 0.25 --reserve-memory 256M api
  • A limit is enforced on the node by the kernel (cgroups). A container that tries to use more memory than its limit is killed by the OOM killer (exit code 137, OOMKilled: true). A container that wants more CPU than its limit is throttled: it runs slower but is not killed.
  • A reservation is a scheduling promise. The scheduler only places a task on a node that still has that much unreserved CPU and memory. Reservations do not cap anything at runtime and do not pre-allocate memory; they only reduce the node's remaining capacity in the scheduler's books.

cpus: "0.50" means half of one CPU's time. Memory accepts suffixes b, k, m, g.

How the scheduler counts

Imagine a node with 4 CPUs and 8 GB. If every task reserves 1 GB, at most eight tasks fit by memory, regardless of what they really use. Tasks with no reservation count as zero, so the scheduler may pile many of them on one node and then the machine runs out of real memory. Therefore: always set a reservation for anything that matters, and set it close to the typical use.

When no node has enough unreserved capacity, the task stays pending with the reason insufficient resources on N nodes. Scaling up then needs another node, not another command (Scaling Services).

flowchart LR
  R["Reservation<br/>(scheduler promise)"] --> P["Placement decision"]
  L["Limit<br/>(kernel cap)"] --> K["Runtime enforcement<br/>OOM kill or CPU throttle"]

Reservations decide where a task goes; limits decide what it may do once there.

Setting good values

  1. Run the service under realistic load and observe it (docker stats on the node, or your monitoring).
  2. Set the reservation near the typical working set (for example the 50th to 75th percentile).
  3. Set the memory limit above the peak with headroom (peak plus 20-30%), since hitting it kills the task.
  4. Set the CPU limit only if you must protect neighbours; many teams reserve CPU but leave the limit generous, because throttling causes latency spikes.
  5. Re-measure after each release.

Languages with their own heap management need extra care: set the JVM's -Xmx or Node's --max-old-space-size below the container limit, otherwise the runtime believes it may use more than the container allows and the OOM killer ends the process. Modern JVMs read the cgroup limit by default (-XX:MaxRAMPercentage), which is the safer setting.

Other per-service controls

YAML
services:
  api:
    ulimits:
      nofile:
        soft: 20000
        hard: 40000
    sysctls:
      net.core.somaxconn: 1024
    deploy:
      resources:
        limits:
          pids: 200

Raising nofile helps servers with many connections. A pids limit stops a fork bomb inside one container. Some keys (such as ulimits) are accepted in stacks, while others may be ignored on older Docker versions; test on your version.

Swarm nodes, not just services

Reserve some capacity for the node itself: the Docker daemon, the OS and the swarm agent need memory too. Never schedule 100% of a node. Managers should run few or no application tasks (docker node update --availability drain manager1 removes tasks from it).

Stand-in: the scheduler's bookkeeping

Python Stand-in: schedule by reservations

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Python Stand-in: OOM kill versus CPU throttling

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Common mistakes

  • Setting only limits, so the scheduler overcommits nodes.
  • Setting reservations equal to peak, which wastes capacity.
  • A memory limit lower than the application's real peak, causing repeated OOM restarts.
  • Forgetting the runtime's own heap setting.
  • Running application tasks on managers and starving the control plane.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does a memory reservation do?

    Reservations affect placement only; they do not cap or pre-allocate.
  2. 2. A container exceeds its memory limit. What happens?

    Memory over the limit triggers an OOM kill (often exit code 137).
  3. 3. A container wants 1 CPU but has a limit of 0.5. What happens?

    CPU limits throttle; they do not kill.
  4. 4. Why set reservations for every important service?

    Without reservations, scheduling ignores real resource needs.

Summary

  • Reservations guide the scheduler; limits are enforced by the kernel at runtime.
  • Exceeding a memory limit kills the task; exceeding a CPU limit only throttles it.
  • Size values from measurements: reservation near typical use, memory limit above peak.
  • Keep headroom on every node and keep application tasks off managers.

Next: Monitoring and Logging a Swarm