Skip to content
Tutorials Center
My learning

Topics

Lesson 18 of 44 ยท Jenkins

Jenkins: Controller, Agents and Jobs

Meet Jenkins: how the controller and agents work, how to run it in Docker, what plugins and credentials do, and how freestyle jobs differ from pipelines.

Jenkins is one of the oldest and most widely deployed automation servers. It is open source, written in Java, and you run it yourself. Where GitHub Actions gives you hosted machines and a YAML file, Jenkins gives you a server you own, a web interface, and a plugin system with thousands of extensions. That flexibility is why large companies still run it, and why you will meet it in many jobs even if you start with a hosted tool.

In this lesson you will learn Jenkins' architecture (the controller and its agents), how to start it in Docker, how plugins and the credentials store fit in, the difference between freestyle jobs and pipelines, and which user interface to use. The real Jenkins cannot run on this site, so commands are read-only. The stand-in is a job queue that dispatches builds to agents by label and executor count, which is the heart of how Jenkins schedules work.

Architecture: controller and agents

A Jenkins installation has two kinds of machines:

  • The controller (older documents call it the master) is the brain. It hosts the web UI, stores job configuration, history and credentials in the Jenkins home directory, schedules builds and talks to agents. There is one controller.
  • Agents (also called nodes) do the work. Each agent offers a number of executors, and each executor can run one build at a time. An agent with three executors can run three builds in parallel.
flowchart LR
    U["developer<br/>web UI / webhook"] --> C["controller<br/>queue, config, credentials"]
    C -- "label: linux" --> A1["agent-1<br/>2 executors"]
    C -- "label: linux, docker" --> A2["agent-2<br/>4 executors"]
    C -- "label: windows" --> A3["agent-3<br/>1 executor"]

The controller keeps the queue and sends each build to an agent that has a matching label and a free executor.

The controller also has a built-in node that can run builds itself. For anything beyond a personal experiment, avoid that: a build runs arbitrary code, and you do not want that code to run on the machine that holds all your credentials and job definitions. The usual recommendation is to give the built-in node zero executors and do all building on agents.

Agents connect to the controller in two ways. With SSH, the controller logs into the agent machine and starts the agent process. With the inbound (formerly JNLP) method, the agent process starts on the agent machine and connects to the controller; this works well when the agent sits behind a firewall, because only an outbound connection is needed. Agents can also be created on demand as containers or cloud instances through plugins, which gives you elastic capacity and clean builds.

Jenkins schedules by labels, much like runs-on in GitHub Actions (see Self-Hosted Runners and Runner Costs). You tag an agent with labels such as linux or docker, and a job can ask for linux && docker.

Running Jenkins in Docker

The quickest way to try Jenkins is the official image. These commands are read-only here, because they need Docker and a browser on your machine:

Terminal
# Start the long-term-support (LTS) release
docker run -d --name jenkins \
  -p 8080:8080 -p 50000:50000 \
  -v jenkins_home:/var/jenkins_home \
  jenkins/jenkins:lts-jdk21

# The first-run unlock password is written to a file and to the log
docker exec jenkins cat /var/jenkins_home/secrets/initialAdminPassword

Port 8080 serves the web interface and 50000 is used by inbound agents. The named volume jenkins_home is essential: all configuration, job history and credentials live in /var/jenkins_home, and without a volume you would lose everything when the container is removed. Open http://localhost:8080, paste the initial admin password, and the setup wizard offers to install a set of suggested plugins and create the first administrator account.

If port 8080 is already used on your machine, map a different host port, for example -p 8081:8080. For a real installation you would put Jenkins behind HTTPS, back up the home directory, and pin a specific image tag rather than following lts. For more about volumes and images, see the Docker tutorial.

Plugins

Out of the box Jenkins knows very little; almost everything is a plugin: source-control integrations (Git), the pipeline engine, credentials types, notification senders, build tools, cloud agents. You manage them under Manage Jenkins > Plugins. A few you will meet again and again:

Plugin What it gives you
Pipeline Pipelines as code with a Jenkinsfile
Git Check out repositories, trigger on changes
Credentials and Credentials Binding Store secrets and expose them to builds safely
Pipeline: Multibranch One job per branch and pull request, discovered automatically
Docker Pipeline Run build steps inside containers

Plugins are Jenkins' strength and its main maintenance burden. Each one is separate software with its own bugs and security advisories, and updates can depend on each other. Good habits: install only what you use, keep the list in version control (for example with a plugin manager file in your Docker image build), update regularly, and test updates on a staging controller before production.

Freestyle jobs and pipelines

Jenkins has two main ways to define work:

  • A freestyle project is configured by filling in forms in the UI: where to get the code, when to build, which shell command to run, what to do afterwards. It is quick for a first experiment. The downside is that the definition lives only inside Jenkins as clicks; you cannot review it in a pull request, copy it easily or see its history in Git.
  • A pipeline is defined as code in a text file called a Jenkinsfile, which usually sits in your repository next to the application. It supports stages, parallel work, conditions and post-build actions, and it can be reviewed and versioned like any other code. This is the style you should use for anything real, and it is the subject of the next lessons.

A tiny pipeline looks like this (read-only; the next lesson explains every line):

GROOVY
pipeline {
  agent { label 'linux' }
  stages {
    stage('Build') {
      steps {
        sh './gradlew build'
      }
    }
  }
}

There are two syntaxes for pipelines, declarative (a structured, opinionated format, shown above) and scripted (free-form Groovy). Most teams start with declarative. You will meet both in this track.

Credentials

Builds need secrets: registry passwords, API tokens, SSH keys. Jenkins stores them in its credentials store (Manage Jenkins > Credentials), where each entry has a type and an ID. Common types include username with password, secret text, secret file, SSH username with private key and certificates. A credential can be scoped globally or to a folder, so only the right jobs can use it.

Jobs never contain the secret; they refer to it by ID and Jenkins injects the value into the build only while it runs, masking it if it appears in the console output. Stored values are encrypted on disk with keys held in the Jenkins home, so protect backups of that directory as carefully as the secrets themselves. Agents, Docker Agents and Credentials covers this in detail.

Classic UI and Blue Ocean

Jenkins ships with a classic web interface. A plugin called Blue Ocean offered a more visual pipeline view with a graphical editor. It is now in maintenance mode: it receives no new features, and new users are generally pointed at the standard Jenkins UI and its Pipeline Graph View style plugins instead. You will still see Blue Ocean in older tutorials and installations, so recognize it, but do not build a new workflow around it. Everything you need can be done in the classic UI and, more importantly, in the Jenkinsfile.

Stand-in: dispatching builds to agents by label

The simulation below models the controller's job: it holds a queue of builds, each requiring a label, and a set of agents with labels and executors. On every tick, the controller starts queued builds on any agent that has a free executor and the required labels, then advances time. It prints a timeline so you can see parallelism, waiting and an unsatisfiable request. It is a model of the idea, not Jenkins' real scheduler.

JS A controller dispatching a build queue to agents

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Notice that at t=0 web-build and api-build both start on agent-1 because it has two executors, and docs-build takes the single executor of agent-2. That means image-build has to wait until t=2, because agent-2 is the only docker-capable agent and it was busy; image-scan queues behind it. Lesson: a general job that grabs a specialized agent can delay the jobs that truly need it, which is why teams reserve special labels for special work. mac-app is queued forever because no agent has the macos label. Jenkins shows that situation in the build queue as a job waiting for an available node.

Nodes, clouds and how agents connect

In Jenkins terms every machine that can run builds is a node: the controller itself (the built-in node) and each agent. A node has a name, a number of executors (how many builds it runs at once), a remote directory and labels that jobs use to ask for it. Agents come in two flavors:

  • Permanent agents: a machine you register once and keep running.
  • Cloud agents: Jenkins asks a cloud (a plugin for Docker, Kubernetes, EC2 and so on) for a fresh agent when a build is queued, and removes it afterwards. Clean machines, no idle cost.

A permanent agent has to connect to the controller somehow. The common launch methods:

Method Who starts the connection Typical use
SSH Controller connects to the agent Linux and macOS machines with an SSH server
WinRM or Windows service Controller or the agent itself Windows machines
Inbound (JNLP/WebSocket) Agent connects out to the controller Agents behind a firewall or in containers

Pick by network direction: if the controller can reach the agent, SSH is simple; if only the agent can reach the controller, use an inbound agent.

Choosing Jenkins

Jenkins is a strong choice when you need full control, run in a private network, have unusual build requirements, or already have a large investment in it. The price is operations work: you upgrade it, patch plugins, back up the home directory and scale the agents. Hosted tools trade that control for convenience. GitHub Actions, Jenkins and GitLab Compared helps you decide.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What is an executor in Jenkins?

    Each agent offers some number of executors. An agent with three executors can run three builds in parallel.
  2. 2. Why mount a volume at /var/jenkins_home when running Jenkins in Docker?

    All Jenkins state lives in its home directory. Without a volume, removing the container deletes jobs, build history and credentials.
  3. 3. Which is the recommended way to define the pipeline for a real project?

    A Jenkinsfile is code: it is versioned, reviewed in pull requests and travels with the project. Freestyle configuration only exists inside Jenkins.
  4. 4. Why should the controller's built-in node usually have zero executors?

    Builds execute arbitrary code. Running them on agents keeps that code away from the controller's secrets and job definitions.
  5. 5. A job requires the label macos but no agent has it. What happens?

    Jenkins keeps the build queued until a node with the right labels and a free executor appears.

Summary

  • Jenkins is a self-hosted automation server: one controller schedules and stores, agents with executors run the builds.
  • Agents connect over SSH or inbound; labels route builds to the right agent, and the controller should not build.
  • The official Docker image runs Jenkins quickly; always mount a volume for /var/jenkins_home.
  • Almost every feature is a plugin; keep the set small and updated.
  • Prefer pipelines (a Jenkinsfile in the repository) over freestyle jobs configured by clicks.
  • Secrets live in the credentials store and are injected by ID. Blue Ocean is in maintenance mode; the classic UI is the default.

Next, you will write a real pipeline: The Declarative Jenkinsfile.