Skip to content
Tutorials Center
My learning

Topics

Lesson 20 of 44 ยท Jenkins

Scripted Pipelines and Shared Libraries

Learn scripted Jenkins pipelines in Groovy: node, parallel, try/catch/finally, then share code with a library (vars/, src/, @Library) and avoid CPS pitfalls.

The declarative Jenkinsfile gives you structure and early validation. Sometimes you need more freedom: loops that build stages from a list, custom retry logic, or code that decides at run time how many parallel branches to start. For that Jenkins offers scripted pipelines, which are ordinary Groovy with a set of pipeline steps added.

When many repositories repeat the same 80 lines of Jenkinsfile, the answer is different again: put the common parts in a shared library and call them from each project. This lesson covers both ideas, plus the one Groovy quirk that surprises almost everyone (CPS).

Jenkins cannot run on this site, so all Groovy below is read-only. The runnable TryIts are stand-ins written in JavaScript that reproduce the semantics (how parallel fails fast, how finally and retry behave). They show the ideas, not the Jenkins engine.

A scripted pipeline

A scripted pipeline has no pipeline { } wrapper. Instead you request an agent with node and write ordinary code inside it:

GROOVY
node('linux') {
    try {
        stage('Checkout') {
            checkout scm
        }
        stage('Build') {
            sh 'make build'
        }
        stage('Test') {
            sh 'make test'
        }
    } catch (err) {
        currentBuild.result = 'FAILURE'
        throw err
    } finally {
        junit allowEmptyResults: true, testResults: 'reports/*.xml'
        cleanWs()
    }
}

Compare this with the declarative version. node('linux') is the equivalent of agent { label 'linux' }, but it is just a step that takes a closure: everything inside runs on that agent and holds an executor until the closure ends. stage is also only a step here, used to group work in the UI. There is no automatic post; you write try/catch/finally yourself, and nothing stops you from forgetting it.

The trade-off is plain: scripted syntax can express anything, and declarative syntax can express a lot but checks more. Teams usually write declarative Jenkinsfiles and reach for scripted code only inside script { } blocks or a shared library.

Setting the result yourself

Jenkins keeps a build result in currentBuild.result. If a step throws, the build fails. You can also catch an error, record it and keep going, which gives you control over "unstable" (yellow) builds:

GROOVY
catchError(buildResult: 'UNSTABLE', stageResult: 'FAILURE') {
    sh './slow-optional-checks.sh'
}

Here the stage is marked failed, but the build is only unstable and the pipeline continues. Other useful steps: retry(3) { ... } repeats a block up to three times, timeout(time: 5, unit: 'MINUTES') { ... } aborts a hung block, and error('message') fails the build on purpose.

parallel

parallel takes a map from branch name to closure and runs the closures at the same time. Because it is plain Groovy, the map can be built in a loop:

GROOVY
def suites = ['unit', 'integration', 'lint']
def branches = [:]
suites.each { name ->
    branches[name] = {
        node('linux') {
            sh "make ${name}"
        }
    }
}
branches.failFast = true
parallel branches

Setting failFast = true in the map tells Jenkins to abort the other branches as soon as one fails. Without it, every branch runs to the end and the build is failed afterwards. Each branch that needs its own machine wraps its work in node, so three branches can use three agents. A branch that only waits (for example on input) should not hold an executor at all.

flowchart TB
    S["parallel"] --> U["unit"]
    S --> I["integration"]
    S --> L["lint"]
    U --> J["join: build result<br/>is worst of all branches"]
    I --> J
    L --> J

Branches start together; the step finishes when all of them have finished or been aborted.

Try it: parallel and fail-fast

Below is a virtual-time model of parallel. Each branch is a list of steps with a duration in seconds. A step that fails ends its branch. With failFast on, the first failure aborts everything else at that moment: steps already running are interrupted, and steps that have not started never do. A finally block runs at the end in both cases.

JS Model parallel with and without failFast

Press Run (or Ctrl + Enter) to see the console output here.

No output.

With failFast = false the lint branch fails at t=2s but unit and e2e still finish (t=6s and t=9s). With failFast = true everything stops at t=2s: unit never starts test, and e2e is interrupted in setup. In both cases the finally runs.

try, catch, finally and retry

The classic pattern is "do the work, always clean up". It works exactly as in any Groovy or Java code, and you rely on it for things like stopping a test database. Next to it sit the Jenkins steps retry, timeout and catchError, which change the result without crashing the whole pipeline.

Try it: retry, catchError and the build result

This model keeps a build result that can only get worse (SUCCESS, then UNSTABLE, then FAILURE). retry re-runs a flaky block, and catchError converts a failure into a worse-but-not-fatal result.

JS Model retry and catchError

Press Run (or Ctrl + Enter) to see the console output here.

No output.

The pipeline keeps going after the flaky step (it succeeded on the third attempt) and after the optional checks, and ends UNSTABLE, not FAILURE.

Shared libraries

A shared library is a Git repository of Groovy code that Jenkins loads into pipelines. It solves duplication: instead of copying a 60-line deployment routine into twenty Jenkinsfiles, you write it once and call it by name. The repository has a conventional layout:

TEXT
my-pipeline-lib/
โ”œโ”€โ”€ vars/
โ”‚   โ”œโ”€โ”€ buildApp.groovy        # global step: buildApp(...)
โ”‚   โ””โ”€โ”€ notifyTeam.groovy      # global step: notifyTeam(...)
โ”œโ”€โ”€ src/
โ”‚   โ””โ”€โ”€ org/acme/Versions.groovy   # regular Groovy classes
โ””โ”€โ”€ resources/
    โ””โ”€โ”€ org/acme/deploy.sh     # files loaded with libraryResource

Each file in vars/ becomes a global step named after the file. By convention it defines a call method, so the file name acts like a function:

GROOVY
// vars/buildApp.groovy
def call(Map config = [:]) {
    String tool = config.tool ?: 'npm'
    sh "${tool} ci"
    sh "${tool} run build"
}

Code in src/ is normal Groovy with packages and classes, good for pure logic such as parsing versions. Code in resources/ can be read as text with libraryResource.

You register the library in Jenkins (name, Git URL, default version) and then load it in a Jenkinsfile with @Library:

GROOVY
@Library('[email protected]') _

pipeline {
    agent any
    stages {
        stage('Build') {
            steps {
                buildApp(tool: 'npm')
            }
        }
    }
    post {
        failure {
            notifyTeam(channel: '#ci')
        }
    }
}

The text after @ is a branch, tag or commit. Pin a tag so that a change to the library cannot break every project at once, and bump it deliberately. The lone underscore after the annotation is required Groovy syntax: an annotation must be attached to something, and _ is a placeholder.

A library is code that every pipeline using it will execute with the pipeline's permissions. Review changes to it with the same care as production code, and restrict who can push to the repository.

CPS: the Groovy quirk

Jenkins runs pipeline Groovy through a transformer called CPS (continuation-passing style). It rewrites your code so a build can be paused, saved to disk and resumed after a controller restart. The price is that some normal Groovy behaves oddly:

  • Objects that are alive across a step call must be serializable. Holding a non-serializable object (a regex Matcher, an open stream) in a variable while calling sh produces a NotSerializableException. Extract what you need into strings first.
  • Closure-heavy Groovy features (custom comparators, certain collection methods, closures that call pipeline steps) have a history of surprising behaviour under CPS. A plain for loop is the safest choice.
  • A method annotated @NonCPS runs as normal Groovy (fast, no pausing), but it cannot call any pipeline step. Use it for pure computation, like parsing JSON text.
GROOVY
@NonCPS
String firstLine(String text) {
    // pure Groovy only: no sh, echo, or other steps in here
    return text.readLines()[0]
}

The pragmatic advice: keep Groovy simple, push heavy logic into shell scripts or compiled tools, and keep pipeline code to orchestration.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. In a scripted pipeline, how do you guarantee a cleanup step runs even when a stage throws?

    post is a declarative feature. Scripted pipelines use ordinary try/catch/finally.
  2. 2. What does setting failFast to true in the map passed to parallel do?

    With failFast, the first failing branch causes the remaining branches to be aborted instead of running to the end.
  3. 3. Which file in a shared library defines a global step that you call as notifyTeam(...)?

    Files in vars/ become global steps named after the file, and the call method is what runs when you use the name like a function.
  4. 4. Why pin a library with @Library('[email protected]') instead of using the default branch?

    A tag gives a stable version. You upgrade pipelines deliberately instead of all at once when the branch moves.
  5. 5. What is a @NonCPS method allowed to do?

    @NonCPS code runs as normal Groovy and cannot be paused, so it must not call pipeline steps.

Summary

  • Scripted pipelines are plain Groovy: node allocates an agent, stage groups work, and you handle failures with try/catch/finally.
  • parallel runs a map of closures; failFast: true aborts siblings after the first failure.
  • retry, timeout and catchError shape the result without crashing the run.
  • A shared library keeps common logic in one repository: vars/ for global steps, src/ for classes, resources/ for files; load it with @Library('name@version') _ and pin a tag.
  • CPS makes pipeline Groovy resumable but restricts it: keep variables serializable and use @NonCPS only for pure code.

Next: Agents, Docker Agents and Credentials.