Skip to content
Tutorials Center
My learning

Topics

Lesson 9 of 40 ยท Pipeline as Code

Common Steps: sh, bat, echo, dir, withEnv

Learn the Jenkins pipeline steps you use daily: sh and bat, returnStdout and returnStatus, dir, withEnv, timeout, retry, sleep, error and unstable, and shell quoting.

A step is one action inside a stage: print a line, run a shell command, change directory, wait, retry. Stages organise the pipeline, but steps are where work happens. Most Jenkinsfiles use the same dozen steps again and again, so learning them well pays off quickly.

The real steps only run on a real Jenkins controller and agent, so they appear in read-only blocks here. The runnable examples are stand-ins: small JavaScript models of how retry, timeout and exit codes behave, so you can watch the logic without a server.

Steps you will use daily

sh / bat Run a shell or batch command.
dir Work in another folder.
withEnv Set variables for a block.
echo Print a line to the log.
1 / 2

echo

echo prints a message to the console log. It is the simplest step and a good way to mark progress.

GROOVY
steps {
  echo 'Starting the build'
  echo "Build number is ${env.BUILD_NUMBER}"
}

Remember the quote rule: single quotes give plain text, double quotes let Groovy replace ${...} before the step runs.

sh and bat

sh runs a command with a shell on a Linux or macOS agent; bat does the same with cmd.exe on Windows (and powershell runs PowerShell). Jenkins starts the shell with -xe: -x prints each command before it runs (that is why logs show lines starting with +), and -e stops the script at the first failing command.

GROOVY
steps {
  sh 'mvn -B clean verify'
  bat 'dir /b'
}

A step fails when the command exits with a non-zero exit code, and a failed step fails its stage and normally the build. That is the contract: exit code 0 means success, anything else means failure.

Multi-line scripts and quoting

Use triple quotes for scripts with several lines. Triple single quotes are not interpolated by Groovy, so $ belongs to the shell:

GROOVY
sh '''
  set -eu
  echo "Home is $HOME"
  for f in *.txt; do
    echo "found $f"
  done
'''

Triple double quotes (""") are interpolated by Groovy. That is handy for ${params.TARGET} but dangerous for shell variables and secrets: Groovy replaces $HOME itself (usually with nothing) before the shell ever sees it. A rule of thumb that avoids most confusion: use single quotes for the script, and pass values in through environment variables.

GROOVY
withEnv(["TARGET=${params.TARGET}"]) {
  sh 'echo "Deploying to $TARGET"'
}

Capturing output and exit status

By default sh just runs. Two options change what it gives back:

GROOVY
script {
  def sha = sh(script: 'git rev-parse --short HEAD', returnStdout: true).trim()
  echo "Short commit: ${sha}"

  def code = sh(script: 'grep -q TODO README.md', returnStatus: true)
  if (code == 0) {
    echo 'README still has TODO notes'
  }
}
  • returnStdout: true returns the command's output as a string. It nearly always ends with a newline, so call .trim().
  • returnStatus: true returns the exit code instead of failing the step. Use it when a non-zero exit is information, not an error (like grep finding nothing).

Because these use variables and if, they sit inside script { }.

dir and withEnv

dir('subfolder') { ... } changes the working directory for the steps inside the block, then goes back. The folder is relative to the workspace and is created if it does not exist. It is much safer than writing cd inside many different sh steps.

GROOVY
dir('frontend') {
  sh 'npm ci'
  sh 'npm run build'
}

withEnv(['NAME=value']) { ... } sets environment variables only for the steps inside the block. Each item is a NAME=value string.

timeout, retry and sleep

GROOVY
timeout(time: 10, unit: 'MINUTES') {
  sh './slow-integration-tests.sh'
}

retry(3) {
  sh './download-dependency.sh'
}

sleep time: 5, unit: 'SECONDS'
  • timeout aborts the block if it runs longer than the limit. The unit can be SECONDS, MINUTES or HOURS.
  • retry(3) runs the block up to three times in total (the first try plus two retries) and fails only if all attempts fail. Use it for steps that fail for reasons outside your control, such as a flaky network, not to hide broken tests.
  • sleep waits. Combine it with retry for a simple pause between attempts.

You can nest them. retry(3) { timeout(time: 2, unit: 'MINUTES') { ... } } gives each attempt its own two-minute limit.

error, unstable and catchError

Sometimes the pipeline itself needs to decide the outcome:

GROOVY
script {
  if (!fileExists('dist/index.html')) {
    error 'The build produced no index.html'
  }
  if (readFile('report.txt').contains('WARN')) {
    unstable 'Report has warnings'
  }
}
  • error 'message' stops the step with a failure.
  • unstable 'message' marks the build UNSTABLE (yellow) but keeps going.
  • catchError(buildResult: 'UNSTABLE', stageResult: 'FAILURE') { ... } lets the block fail without stopping the rest of the pipeline, while recording the result you choose. warnError('message') { ... } is a shortcut that marks the build unstable.

The script step

script { } opens a block of full Groovy inside a declarative steps section. Keep it short; if you find yourself writing dozens of lines, move the logic into a shell script in the repository or into a shared library. More on writing whole pipelines in Groovy in Scripted Pipeline and Groovy for Jenkins.

Try the logic: exit codes and retry

A step succeeds when a command exits with 0. Here is a model of sh with returnStatus, and of retry around a flaky command that fails twice and then works. The "commands" are plain functions returning an exit code.

JS A stand-in for sh, returnStatus and retry

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Notice the difference: with returnStatus the failing grep does not stop anything, while a plain sh throws. The third run keeps failing, so after three attempts the error escapes, which is exactly how a real retry(3) behaves.

Try the logic: timeout

A real timeout uses the clock. To keep this example deterministic, the stand-in uses virtual time: each task reports how long it would take, and the wrapper compares that with the limit.

JS A stand-in for timeout and sleep

Press Run (or Ctrl + Enter) to see the console output here.

No output.

A real timeout interrupts the running step the moment the limit passes, rather than waiting for the block to finish as this model does. The numbers and the decision are the same, though.

Habits worth building

  • Put every long or risky step inside a timeout.
  • Keep sh scripts short and call a script file from the repository for anything bigger. Scripts in the repository can be run and tested on a laptop.
  • Use set -eu (and set -o pipefail in bash) in multi-line shell scripts so that errors are not ignored.
  • Never echo a secret; see Credentials and Secrets.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. A sh step ends with exit code 2. What happens by default?

    Any non-zero exit code fails the step. Use returnStatus: true if you want to receive the code instead.
  2. 2. Why do you usually call .trim() on the result of sh(returnStdout: true)?

    Command output normally ends with a newline, which would break comparisons and file names.
  3. 3. What does retry(3) do?

    retry(3) means three attempts in total: the first one plus up to two repeats.
  4. 4. In a triple-single-quoted sh script, who expands $HOME?

    Single quotes stop Groovy from interpolating, so the dollar sign reaches the shell, which expands it.
  5. 5. Which step marks the build yellow without stopping it?

    unstable sets the result to UNSTABLE and the pipeline continues.

Summary

  • sh and bat run commands; a non-zero exit code fails the step.
  • returnStdout returns output (trim it), returnStatus returns the exit code instead of failing.
  • Single-quoted scripts leave $ to the shell; double-quoted ones are interpolated by Groovy first.
  • dir and withEnv scope a folder or variables to a block.
  • timeout, retry and sleep control timing; error, unstable and catchError decide the result.
  • Wrap real Groovy code in script { } and keep it short.

Next, organise steps into stages that run side by side in Stages, Parallel and Matrix.