Skip to content
Tutorials Center
My learning

Topics

Lesson 15 of 38 ยท Services and Tasks

Resource Limits and Reservations

Set CPU and memory limits and reservations on Swarm services, understand how the scheduler uses reservations and what happens when a container exceeds its limit.

Containers share the machine. Without boundaries, one memory leak can make the kernel kill unrelated containers, and one busy loop can starve its neighbours. Swarm lets you set two different kinds of numbers per service: limits, which cap what a container may use, and reservations, which tell the scheduler how much room a task needs. Mixing them up is the most common mistake in this area.

Limits: a ceiling enforced on the node

CONSOLE
$ docker service create --name api \
    --limit-cpu 0.5 \
    --limit-memory 256M \
    myorg/api:1
  • --limit-cpu 0.5 means at most half of one CPU core's time (it is a quota, not a pinning). 2 means two cores' worth.
  • --limit-memory 256M is a hard cap. When the container tries to allocate beyond it, the kernel's OOM killer terminates a process in the container; the task fails with exit code 137 and Swarm replaces it according to the restart policy.
  • CPU limits throttle (the container is slowed); memory limits kill.

Limits are enforced by the node's kernel (cgroups). The scheduler does not use them for placement.

Reservations: a promise to the scheduler

CONSOLE
$ docker service create --name api \
    --reserve-cpu 0.25 \
    --reserve-memory 128M \
    myorg/api:1

A reservation says "this task needs at least this much". The scheduler only places the task on a node with enough unreserved capacity, and counts the reservation as used from then on. If no node has room, the task stays pending with "insufficient resources".

Reservations do not guarantee the memory physically (they are accounting for placement, plus a soft hint for the CPU share), and they do not cap usage. They make sure you do not pile ten hungry tasks on one small node.

Using both together

CONSOLE
--reserve-cpu 0.25 --limit-cpu 1 --reserve-memory 128M --limit-memory 256M
Reservation Limit
Used by Scheduler Node kernel
Effect Task needs this much free to be placed Task can never exceed this
CPU exceeded n/a Throttled
Memory exceeded n/a OOM kill, task fails

Practical guidance:

  • Always set memory limits for services you do not fully trust to behave, and a reservation near typical usage.
  • Set the reservation to what the service normally needs and the limit to a safe peak. A big gap means you can overcommit the node: reservations fit, but peaks may collide.
  • Setting reservation = limit gives predictable, "guaranteed-style" containers at the cost of lower density.
  • Java and other runtimes that size the heap from the container limit need a limit set, or they will assume the whole machine.

What the scheduler sees

The node's total CPUs and memory appear in docker node inspect. The sum of reservations of tasks already assigned to a node is subtracted from it. You can read a service's numbers:

CONSOLE
$ docker service inspect --format '{{json .Spec.TaskTemplate.Resources}}' api
$ docker node inspect --format '{{.Description.Resources}}' worker1

Live usage per container is shown by docker stats on each node (swarm has no cluster-wide stats; use a monitoring tool such as cAdvisor with Prometheus, see Operations).

Other limits

  • --limit-pids caps the number of processes (protects against fork bombs).
  • --ulimit nofile=65536:65536 sets file descriptor limits.
  • --generic-resource lets you declare custom resources such as gpu=1 that the scheduler counts; nodes advertise them in daemon.json.

Updating

CONSOLE
$ docker service update --limit-memory 512M api

A change triggers a rolling update, as for any spec change.

Stand-in: the scheduler's capacity check

The simulator tracks reservations per node and shows how tasks are accepted or left pending. This is the same idea as the real scheduler's resource filter, simplified to memory.

Python Stand-in: reservations and capacity

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Two nodes with 1024 MB hold exactly four 512 MB reservations; the fifth task waits.

Stand-in: what a limit does

This tiny model contrasts the two limit types: CPU is throttled, memory kills.

Python Stand-in: limit behaviour

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Check your understanding

Quick check ยท 3 questions

of 3 answered You scored / 3 ยท perfect!

  1. 1. What does --reserve-memory 128M influence?

    Reservations are for scheduling: the task needs a node with at least this much unreserved memory.
  2. 2. A container exceeds --limit-memory. What happens?

    Memory limits are hard; CPU limits only throttle.
  3. 3. Tasks are pending with 'insufficient resources'. A likely cause is:

    The scheduler cannot find a node with enough unreserved CPU or memory.

Summary

  • Limits cap usage on the node; reservations drive placement.
  • CPU over the limit is throttled; memory over the limit kills the container.
  • Reserve typical use, limit at a safe peak, and always limit memory.
  • Use --limit-pids and custom generic resources where needed.

Next: Health Checks and Restart Policies