Skip to content
Tutorials Center
My learning

Topics

Lesson 10 of 40 ยท Pipeline as Code

Stages, Parallel and Matrix

Organise a Jenkins pipeline into sequential stages, run branches in parallel with failFast, expand a matrix of axes with excludes, and nest stages clearly.

Stages split a pipeline into named phases such as Build, Test and Deploy. By default they run one after another, and a failure stops the line. Two features change that: parallel runs several stages at the same time to save wall-clock time, and matrix repeats the same stages for every combination of values (for example every operating system with every Node version).

None of this can run on a web page, so the real syntax is read-only. The stand-ins model the scheduling: how long a parallel group takes, when failFast cancels siblings, and how a matrix expands into cells.

Three ways to arrange stages

Sequential One after another, stop on failure.
Parallel Branches run at the same time.
Matrix Every combination of axes.

Pick the simplest layout that gives you fast feedback.

1 / 2

Sequential stages

Every stage in stages { } runs in order. A stage that fails marks the build failed, and the later stages are skipped (stages with post { always } and the pipeline-level post still run).

GROOVY
pipeline {
  agent any
  stages {
    stage('Build')  { steps { sh 'make build' } }
    stage('Test')   { steps { sh 'make test' } }
    stage('Package'){ steps { sh 'make package' } }
  }
}

The stage view in the web interface draws each stage as a column and colours it green, red, yellow (unstable) or grey (skipped), so the names you choose are what your team sees. See A Tour of the Jenkins Interface.

Parallel branches

Put a parallel block inside a stage and give it nested stages. Each nested stage is one branch.

GROOVY
stage('Checks') {
  failFast true
  parallel {
    stage('Unit tests') {
      steps { sh 'make unit' }
    }
    stage('Lint') {
      steps { sh 'make lint' }
    }
    stage('Security scan') {
      agent { label 'scanner' }
      steps { sh 'make scan' }
    }
  }
}

Points to know:

  • A stage contains either steps, stages, parallel or matrix; never two of them.
  • Each branch can have its own agent, environment, when and post. Branches without an agent share the stage's agent and workspace, so two branches writing the same file will collide. Give branches separate directories or separate agents when they write files.
  • Branches need free executors to run at once. If only one executor is free, they take turns. See Jenkins Architecture.
  • The overall stage finishes when its slowest branch finishes.

failFast

By default every branch runs to its end even if one fails, so you see all failures in one build. With failFast true on the stage (or parallelsAlwaysFailFast() in the pipeline options), the first failing branch aborts the others. Use it when a failure makes the rest of the work worthless and you want to free executors fast.

Nested sequential stages

Inside a parallel branch you sometimes need several steps in order, each shown as its own box. Use stages for that:

GROOVY
stage('Platforms') {
  parallel {
    stage('Linux') {
      stages {
        stage('Build')   { steps { sh 'make' } }
        stage('Test')    { steps { sh 'make test' } }
      }
    }
    stage('Windows') {
      agent { label 'windows' }
      stages {
        stage('Build')   { steps { bat 'build.bat' } }
        stage('Test')    { steps { bat 'test.bat' } }
      }
    }
  }
}

Keep nesting shallow. The pipeline syntax allows only certain combinations; if you nest too deeply, the validator tells you what is not allowed (see Declarative Pipeline Syntax in Depth).

Matrix

A matrix repeats a set of stages for every combination of axis values. Each combination is called a cell, and cells run in parallel.

GROOVY
stage('Test matrix') {
  matrix {
    axes {
      axis {
        name 'PLATFORM'
        values 'linux', 'windows', 'mac'
      }
      axis {
        name 'NODE_VERSION'
        values '20', '22'
      }
    }
    excludes {
      exclude {
        axis { name 'PLATFORM'; values 'mac' }
        axis { name 'NODE_VERSION'; values '20' }
      }
    }
    stages {
      stage('Test') {
        steps {
          echo "Testing on ${PLATFORM} with Node ${NODE_VERSION}"
        }
      }
    }
  }
}

Three platforms times two versions is six cells; the exclude removes one, leaving five. Inside the cell, each axis name is available as an environment variable. A matrix can also have its own agent (so each cell may run on a different machine, such as agent { label "${PLATFORM}" }), and a failFast true setting works here too.

Matrices multiply quickly. Four axes with four values each is 256 cells. Because each cell needs an executor, large matrices can flood the queue; keep the axes to values that really differ.

Model: how long does a parallel group take?

This stand-in schedules branches onto a limited number of executors using virtual minutes. It answers the practical question: how much do I save, and does a small agent pool erase the gain?

JS Scheduling parallel branches

Press Run (or Ctrl + Enter) to see the console output here.

No output.

With one executor, parallel gives no gain (8 minutes); with three, the group is as slow as its slowest branch (4 minutes). That is why parallel stages and agent capacity must be planned together; see Agents, Labels and Scaling.

Model: failFast and matrix expansion

The second stand-in expands a matrix with an exclude, then runs the cells in virtual time. One cell fails; with failFast the still-running cells are cancelled at that moment.

JS Matrix expansion and failFast

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Without failFast every cell runs to its end, so you learn about all problems. With it, the slower cells stop as soon as the first failure appears (minute 4 here), which frees executors sooner but hides what else might have failed.

Choosing a layout

  • Start sequential. It is the easiest to read and debug.
  • Move independent, slow checks (tests, lint, scans) into parallel.
  • Use matrix when you really must prove the same thing on several platforms or versions.
  • Prefer failFast for expensive pipelines on pull requests, and leave it off on nightly builds where you want the full picture.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. How long does a parallel stage take when there are enough free executors?

    Branches run at the same time, and the stage ends when the last of them finishes.
  2. 2. What does failFast true do?

    failFast cancels the remaining branches after the first failure, freeing executors early.
  3. 3. A matrix has PLATFORM with 3 values and NODE_VERSION with 2 values, and one exclude. How many cells run?

    3 x 2 = 6 combinations, and the exclude removes one, leaving 5.
  4. 4. Which is NOT allowed in a single stage?

    A stage contains exactly one of steps, stages, parallel or matrix.

Summary

  • Stages run in order by default; a failure stops later stages.
  • parallel runs nested stages at once; it needs free executors to help, and branches must not fight over the same workspace files.
  • failFast aborts siblings after the first failure.
  • matrix expands axes into cells and excludes remove combinations; watch the multiplication.
  • A stage holds exactly one of steps, stages, parallel or matrix.

Next, learn how values flow into those stages in Environment Variables and Parameters.