Skip to content
Tutorials Center
My learning

Topics

Lesson 3 of 40 ยท Meet Jenkins

Jenkins Architecture: Controller, Agents and Executors

Understand how Jenkins is built: the controller, agents and executors, workspaces, the JENKINS_HOME folder, the build queue and the life of a build.

Before you create jobs, it pays to know what the pieces of Jenkins are and how a build travels through them. Most Jenkins problems ("my build is stuck", "the disk is full", "the wrong tool version ran") are architecture problems in disguise. Once you can name the parts, those problems become easy to reason about.

Jenkins uses a controller and agents design. One server decides what runs and keeps all the records; one or many worker machines actually run the builds. (Older material calls them "master" and "slave". The current terms are controller and agent.) The real server cannot run on this site, so the diagrams and configuration here are read-only, and the runnable examples are stand-ins written in Node that model the scheduling.

The cast of characters

Controller Holds config, schedules builds, shows the UI.
Agent A machine that runs build steps.
Executor One build slot on a machine.
Workspace The folder where a build keeps its files.

Controller decides, agents work.

1 / 2

The controller

The controller is the Jenkins server process (a Java application). It:

  • serves the web interface and the remote API,
  • stores every job definition, credential, plugin and build record,
  • watches for triggers and puts work in the build queue,
  • chooses an agent and an executor for each queued item,
  • collects logs and results from the agents.

A common rule of thumb: the controller should orchestrate, not build. Running compilers and test suites on the same machine that holds all your secrets and configuration is slow and risky, because the code under test is untrusted in the security sense. Best practice is to set the number of executors on the controller to 0 and give every job a proper agent.

Agents and executors

An agent is any machine (physical, virtual, a container or a cloud instance) that can run build steps for the controller. Agents are described by a few properties:

  • Label(s), such as linux, java21 or gpu. A job can ask for a label, and Jenkins picks a matching agent.
  • Number of executors, the number of builds that can run at once on that agent.
  • A remote root directory where workspaces live.

An executor is a single slot. A machine with 4 executors can run 4 builds in parallel. More executors are not always better: if each build is CPU hungry, 4 on a 2-core machine just makes everything slow. A common starting point is one executor per CPU core, adjusted by watching real load.

An agent connects to the controller in one of two ways:

  • Controller starts it (outbound). Jenkins logs into the machine over SSH and launches a small program called the agent. The machine needs Java and an open SSH port.
  • Agent starts it (inbound). The agent machine runs a command that connects to Jenkins. This suits agents behind firewalls, in cloud autoscaling, or in containers, because the controller does not need to reach into them.

Both use a protocol called remoting, a channel over which the controller sends work and the agent streams back output. You do not write anything for it; you only need to know that the agent must be able to reach (or be reached by) the controller.

A read-only example of starting an inbound agent from the agent machine looks like this:

Terminal
# Run on the agent machine (the secret and URL come from the node's page in Jenkins)
java -jar agent.jar \
  -url https://jenkins.example.com/ \
  -name build-agent-1 \
  -secret <the-secret-shown-in-jenkins> \
  -workDir /home/jenkins/agent

Where Jenkins keeps its data: JENKINS_HOME

Everything the controller knows is stored as plain files in one directory, called JENKINS_HOME. There is no separate database to install, which makes backups simple: copy the folder.

TEXT
JENKINS_HOME/
  config.xml            global settings
  credentials.xml       encrypted credentials
  secrets/              encryption keys (guard these!)
  plugins/              installed plugins
  jobs/
    my-app/
      config.xml        the job definition
      builds/
        1/              build #1: log, build.xml, artifacts
        2/
      nextBuildNumber
  nodes/                agent definitions
  workspace/            workspaces for builds on the controller
  users/                user accounts and settings

Two practical consequences. First, JENKINS_HOME can grow large: every build keeps its log and artifacts until a discard rule removes them (see Artifacts, Stash and Fingerprints). Second, the folder contains encryption keys and credentials, so it needs the same care as a password vault.

The life of a build

Here is what happens when a build is triggered:

flowchart LR
  T["Trigger<br/>(push, schedule, click)"] --> Q["Build queue"]
  Q --> S{"Free executor<br/>with matching label?"}
  S -- "no" --> Q
  S -- "yes" --> W["Workspace created<br/>or reused"]
  W --> R["Steps run on the agent"]
  R --> L["Log and result sent<br/>to the controller"]
  L --> D["Build recorded<br/>(SUCCESS, FAILURE, ...)"]

Items wait in the queue until an executor with the right label is free.

Important details:

  • The queue is the waiting room. If a job asks for the label windows and no Windows agent is online, the item waits, and the UI says why.
  • A workspace is a directory on the agent, usually one per job. Builds of the same job on the same agent may reuse it, which is fast but means leftovers from earlier builds can still be there. Many pipelines clean it at the start.
  • The result is one of SUCCESS, UNSTABLE (built, but tests failed), FAILURE, ABORTED or NOT_BUILT.

A stand-in scheduler

Below is a simplified scheduler written in Node. It is not Jenkins' real algorithm; it shows how a queue, labels and executors combine. Two agents have different labels and executor counts. Builds are matched in order, and a build with no free matching executor waits in the queue.

Node.js A stand-in queue and executor scheduler

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

The first three items start, docs waits because both Linux slots are taken, and signer waits because the only Windows executor is busy. When api-tests finishes, docs takes its slot. Notice that a windows build never moved onto Linux; labels are a hard requirement.

Checking the layout yourself

A second stand-in turns the JENKINS_HOME listing above into data and answers two practical questions: how many builds does a job keep, and which job will use the most disk?

Node.js A stand-in JENKINS_HOME disk report

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

A single chatty job with 365 retained builds dominates the disk. A build-discard policy, which you will set in Artifacts, Stash and Fingerprints, is the cure.

When you diagnose "my build does not start", read the queue entry first. Jenkins prints the reason, such as "Waiting for next available executor" or "There are no nodes with the label 'x'".

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What should the controller mainly do?

    The controller coordinates and records. Builds should run on agents, which keeps secrets safer and the server responsive.
  2. 2. What is an executor?

    A node with N executors can run N builds in parallel. Each running build occupies one executor.
  3. 3. Which agent connection style works best when the agent is behind a firewall?

    An inbound agent opens the connection itself, so the controller does not need to reach into the private network.
  4. 4. Where does Jenkins store job definitions and build records?

    Jenkins keeps everything as files under JENKINS_HOME, so backing up that directory (carefully) backs up Jenkins.

Summary

  • The controller schedules and records; agents run the steps. Keep builds off the controller.
  • An executor is one build slot; labels decide which agents may run a job.
  • Agents connect outbound (SSH from the controller) or inbound (the agent calls the controller) using remoting.
  • JENKINS_HOME holds all configuration, credentials, plugins and build history as files.
  • A build moves from trigger to queue to executor to workspace to recorded result.

Next, set up your own server in Installing Jenkins.