Skip to content
Tutorials Center
My learning

Topics

Lesson 41 of 111 ยท Containers in Depth

GPUs and Devices in Containers

Give containers access to host hardware: --device, device cgroup rules, --gpus with the NVIDIA Container Toolkit, CDI, Compose GPU reservations and AMD or Intel GPUs.

A container normally sees a tiny, safe /dev folder: null, zero, random, a terminal and a few more. That is exactly what you want for a web server, and exactly what you do not want for a program that trains a neural network, talks to a USB serial adapter, reads a webcam or plays sound. Those programs need access to a piece of hardware that belongs to the host.

Docker has two tools for that. --device hands one device file to a container. --gpus is a higher-level switch that prepares a whole GPU for use, including its driver libraries. This lesson explains both, how the NVIDIA Container Toolkit fits in, how to ask for GPUs in Compose, and the security price of each option. Commands were run on Docker Engine 29.8 on a Windows machine with no usable NVIDIA GPU, so the GPU parts are read-only examples; the --device experiments are real and ran in a Linux container.

Devices are files

On Linux, hardware is exposed as device files under /dev. A device file has a type (c for character devices, b for block devices) and two numbers, major and minor, that tell the kernel which driver and which unit it refers to. /dev/null is a character device with numbers 1 and 3.

A container gets its own /dev, filled with a minimal set. Count it, and compare with a container started in privileged mode:

CONSOLE
$ docker run --rm alpine:3.22 sh -c 'ls /dev | tr "\n" " "'
core fd full mqueue null ptmx pts random shm stderr stdin stdout tty urandom zero
$ docker run --rm alpine:3.22 sh -c 'ls /dev | wc -l'
15
$ docker run --rm --privileged alpine:3.22 sh -c 'ls /dev | wc -l'
157

An ordinary container sees 15 entries. A --privileged container sees 157, because Docker exposes every device of the (virtual) host to it. That is why privileged mode is a last resort and why you should add only what you need. See Capabilities and Non-Root for the security side.

flowchart TD
    subgraph Host
        D1["/dev/ttyUSB0"]
        D2["/dev/dri/renderD128"]
        D3["/dev/nvidia0"]
        K["Kernel drivers"]
    end
    D1 --> K
    D2 --> K
    D3 --> K
    C["Container<br/>own /dev with 15 entries"]
    D1 -- "--device" --> C
    D3 -- "--gpus" --> C

Without flags the container sees none of the host's devices. --device and --gpus add specific ones.

--device: pass one device through

The syntax is --device host-path[:container-path[:permissions]]. The permissions are any of r (read), w (write) and m (mknod). To avoid touching real hardware in the example, map /dev/null under a new name:

CONSOLE
$ docker run --rm --device /dev/null:/dev/mynull alpine:3.22 sh -c 'ls -l /dev/mynull; echo hi > /dev/mynull; echo wrote ok'
crw-rw-rw-    1 root     root        1,   3 Oct 29 10:18 /dev/mynull
wrote ok

The file /dev/mynull is a character device with numbers 1, 3, the same as /dev/null on the host. With a real device the command looks the same, for example a serial adapter:

CONSOLE
$ docker run --rm -it --device /dev/ttyUSB0 my-image minicom -D /dev/ttyUSB0

If the device does not exist on the host, Docker refuses to start the container instead of failing later:

CONSOLE
$ docker run --rm --device /dev/nonexist alpine:3.22 true
docker: Error response from daemon: error gathering device information while adding custom device "/dev/nonexist": no such file or directory

If you use Git Bash on Windows, it rewrites arguments that look like Unix paths (/dev/null becomes C:/Program Files/Git/dev/null). Set MSYS_NO_PATHCONV=1 for the command, or use PowerShell. Devices also only exist inside Docker's Linux virtual machine on Docker Desktop, so USB devices need extra steps there (for example the usbipd tool for WSL 2).

Permissions and cgroup rules

By default a container may not create arbitrary device files and may only use the devices Docker lists for it. Two related options exist for special cases:

  • --device-cgroup-rule='c 189:* rmw' allows access to all devices with major number 189 (USB devices), including ones plugged in after the container started. Without it, --device only covers devices present at start time. This is a read-only example for a USB hot-plug scenario.
  • --cap-add MKNOD lets the container create device nodes. It is part of the default capability set already, so you rarely add it, but you may need to combine it with a cgroup rule.

Together they allow a container that creates its own device nodes for devices that appear later, but they are the sharp end of the toolbox. Prefer an explicit --device for fixed hardware.

Common device recipes

Hardware Typical flags
Serial or USB-serial adapter --device /dev/ttyUSB0 (add --group-add dialout if the user is not root)
Webcam --device /dev/video0
Sound card --device /dev/snd
Intel or AMD GPU (rendering, video encoding) --device /dev/dri
AMD GPU for ROCm compute --device /dev/kfd --device /dev/dri --group-add video
FUSE file systems --device /dev/fuse (plus the right capability)

The device file is owned by a host group such as dialout or video. Inside the container, a non-root user needs a matching group id, which --group-add provides. If you see permission denied on an existing device, check the group first (see Users and Permissions).

--gpus: GPUs for compute

A GPU is more than one device file. A program also needs the matching driver libraries (for example the CUDA driver library libcuda.so) at the same version as the kernel driver on the host. Putting those libraries into every image would make images depend on the host. The NVIDIA Container Toolkit solves this: it hooks into container start-up and mounts the right device files and driver libraries from the host into the container. Docker's --gpus flag turns that on.

What you need on the host (Linux):

  1. A recent NVIDIA driver (check with nvidia-smi on the host).
  2. Docker Engine.
  3. The NVIDIA Container Toolkit, registered as a runtime. The toolkit's command nvidia-ctk runtime configure --runtime=docker edits the daemon configuration, then you restart Docker.

You can see whether the runtime is registered:

CONSOLE
$ docker info --format '{{range $k,$v := .Runtimes}}{{$k}} {{end}}'
io.containerd.runc.v2 nvidia runc

On Docker Desktop for Windows with the WSL 2 backend, NVIDIA GPUs are supported by installing the normal Windows driver; no Linux driver goes inside WSL. Docker Desktop for macOS cannot pass a GPU through to Linux containers.

With everything set up, these are the typical uses (read-only examples, since this machine has no GPU):

CONSOLE
$ docker run --rm --gpus all nvidia/cuda:12.6.0-base-ubuntu24.04 nvidia-smi
$ docker run --rm --gpus 2 my-training-image python train.py
$ docker run --rm --gpus '"device=0,2"' my-training-image python train.py
$ docker run --rm --gpus all,capabilities=utility,compute my-image nvidia-smi

The values mean: all exposes every GPU, a number picks that many GPUs, and device= picks specific ones by index or UUID (the odd quoting protects the comma from the shell). capabilities limits which features the container receives; compute and utility are common, and graphics or video appear for rendering and transcoding.

On a machine without a working GPU the failure is clear, and I could reproduce it:

CONSOLE
$ docker run --rm --gpus all alpine:3.22 true
docker: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error running prestart hook #0: exit status 1, stdout: , stderr: Auto-detected mode as 'legacy'
nvidia-container-cli: initialization error: WSL environment detected but no adapters were found

The error comes from the toolkit's hook (nvidia-container-cli), not from Docker itself: the GPU request reached the runtime, and the hook found no adapter. The messages vary: could not select device driver "" with capabilities: [[gpu]] means the toolkit is not installed or not registered at all, and a driver/library mismatch error usually means the host driver was updated without restarting the container or the machine.

Your image only needs the CUDA user-space libraries your application uses (a PyTorch or TensorFlow image brings its own), never the kernel driver. Base images from NVIDIA named nvidia/cuda:<version>-base, -runtime or -devel are the common starting points; see Choosing Base Images.

CDI: a newer way to name devices

The Container Device Interface (CDI) is an open specification that describes devices in files, so any runtime can inject them without vendor-specific hooks. Recent Docker Engine versions can use CDI names directly, and the NVIDIA toolkit can generate the description:

CONSOLE
$ sudo nvidia-ctk cdi generate --output=/etc/cdi/nvidia.yaml
$ docker run --rm --device nvidia.com/gpu=all ubuntu:24.04 nvidia-smi

This is a read-only example. CDI is where the ecosystem is heading (Podman and Kubernetes use it too), but --gpus remains the most common form in tutorials and CI scripts. Check which one your Docker version documents as supported.

GPUs in Compose

In a Compose file you ask for GPUs as a resource reservation:

YAML
services:
  trainer:
    image: my-training-image
    command: python train.py
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1          # or: all, or device_ids: ["0"]
              capabilities: [gpu]

count and device_ids are alternatives. capabilities: [gpu] is required. Plain device files use the short devices: key of a service, for example devices: ["/dev/ttyUSB0:/dev/ttyUSB0"]. See Compose Services for the rest of the service options.

Security and best practices

  • Never use --privileged to get one device. It disables most isolation, as the 157-versus-15 count shows. Use the exact --device.
  • A device file is a direct line into a kernel driver. A bug in the driver is reachable from the container, so grant devices only to images you trust.
  • GPUs are shared hardware. Without limits, one container can use all GPU memory; use --gpus device=N to give each job its own card, and use the vendor's tools (MIG on supported NVIDIA cards, for example) for finer slicing.
  • Pin the CUDA image tag to the version your framework needs, and keep the host driver recent enough for it. The driver must be at least as new as the CUDA version inside the image requires.
  • Test the plain case first: nvidia-smi in a container tells you in seconds whether the toolkit and the driver are working before you debug your own code.
  • On shared CI runners, pass devices through Compose or the job configuration only for the jobs that need them.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. In the experiment, an ordinary container showed 15 entries in /dev and a --privileged container showed 157. What does that tell you?

    A privileged container receives the host's devices. A normal container gets only a minimal /dev, plus whatever you add with --device.
  2. 2. What does --device /dev/ttyUSB0 do?

    --device adds that single device file to the container's /dev. It does not install anything and does not grant privileges.
  3. 3. Why does --gpus need the NVIDIA Container Toolkit instead of just --device?

    Device files alone are not enough. User-space driver libraries such as libcuda must match the host driver version, so the toolkit mounts them from the host.
  4. 4. You run docker run --gpus all on a machine without the toolkit. Which message is typical?

    Without the registered nvidia runtime Docker cannot satisfy the GPU request and stops with an error.

Summary

  • A container gets a minimal /dev (15 entries in our test); --privileged exposes everything (157), so avoid it.
  • --device host[:container[:rwm]] passes a single device file; add --group-add when a non-root user needs the device's group, and --device-cgroup-rule for hot-plugged USB devices.
  • --gpus relies on the NVIDIA Container Toolkit, which mounts device files and matching driver libraries; options are all, a count, or device= lists, plus capabilities.
  • CDI (--device vendor.com/class=name) is the vendor-neutral successor; AMD and Intel GPUs use /dev/dri (and /dev/kfd for ROCm).
  • In Compose, GPUs are deploy.resources.reservations.devices with driver: nvidia.
  • Give only the devices a job needs and test with nvidia-smi first.

Next: Data and Volumes