Skip to content
Tutorials Center
My learning

Topics

Lesson 17 of 40 ยท Build, Test and Package

Building Java, Node and Python Projects

Build Java, Node.js and Python projects in Jenkins: the tools directive, Maven, Gradle, npm, pip with virtualenv, dependency caching and toolchains on agents.

Jenkins does not compile anything itself. It runs the tools your project already uses: Maven or Gradle for Java, npm for JavaScript, pip for Python. The skill is in making those tools available on the right agent, in the right version, and fast enough that people do not stop waiting for the build.

This lesson shows how to run the three most common toolchains from a Jenkinsfile, how to choose tool versions, and how to cache downloaded dependencies. Real builds need a Jenkins server and agents, so all pipeline code here is read-only. The runnable example is a stand-in that computes a cache key the way many caching schemes do. For the languages themselves, see the Java, JavaScript and Python tutorials.

Where do tools come from?

Pre-installed Baked into the agent image.
tools directive Jenkins installs a named version.
Container The build runs inside a tool image.
Wrapper mvnw, gradlew and friends.

Whatever you choose, pin the version.

1 / 2

Choosing tool versions

There are four common ways to get a specific version of a build tool onto the machine that runs the step.

1. The tools directive. In Manage Jenkins, under tools, an administrator defines named installations, such as a Maven called maven-3.9 or a JDK called jdk21. A Declarative Pipeline then asks for them by name, and Jenkins puts them on the PATH for that stage or pipeline:

GROOVY
pipeline {
  agent any
  tools {
    maven 'maven-3.9'
    jdk 'jdk21'
  }
  stages {
    stage('Build') {
      steps { sh 'mvn -version' }
    }
  }
}

The tools block supports the installations your plugins provide (Maven, Gradle, JDK, and Node.js through the NodeJS plugin, for example). The names must match what the administrator configured, or the build fails before it starts.

2. A wrapper script in the repository. Maven and Gradle projects commonly ship mvnw and gradlew, small scripts that download the exact version the project wants. This is often the most reproducible option, because the version is decided by the repository, not by whichever Jenkins administrator happened to set up the agent.

3. A container as the agent. The build runs inside a Docker image that already contains the tools. See Docker Agents and Building Images.

4. Tools baked into the agent image. Fast and simple for static agents, but you must keep every agent in sync. Use labels to route a job to agents that have what it needs (see Agents, Labels and Scaling).

Java with Maven and Gradle

Use batch mode and avoid noisy progress output in CI. A typical Maven pipeline:

GROOVY
pipeline {
  agent { label 'linux' }
  tools { jdk 'jdk21' }
  options { timeout(time: 20, unit: 'MINUTES') }
  stages {
    stage('Build and test') {
      steps {
        sh './mvnw -B -ntp clean verify'
      }
    }
  }
}

-B is batch mode, and -ntp hides the transfer progress lines that otherwise fill the log with thousands of rows. verify runs the build up to the integration tests and checks. The Gradle version is similar:

GROOVY
sh './gradlew --no-daemon --console=plain build'

On short-lived agents the Gradle daemon is wasted effort, which is why --no-daemon is common; on a long-lived agent the daemon can speed up repeated builds. Both tools put their test reports in known places, which you will pick up in Tests, JUnit Reports and Quality Gates.

Node.js with npm

The NodeJS plugin lets you declare nodejs 'node-22' in tools, or you can run the build in a Node container. The most important habit is npm ci instead of npm install in CI. It installs exactly what the lock file says, fails if package.json and the lock file disagree, and starts from a clean node_modules:

GROOVY
pipeline {
  agent { docker { image 'node:22-alpine' } }
  stages {
    stage('Install') { steps { sh 'npm ci' } }
    stage('Lint and test') {
      steps {
        sh 'npm run lint'
        sh 'npm test'
      }
    }
    stage('Build') { steps { sh 'npm run build' } }
  }
}

Commit the lock file (package-lock.json), or your builds will drift as new versions of dependencies are released.

Python with pip and virtualenv

Never install packages into the system Python of a shared agent: one project's requirements can break another's. Create a virtual environment inside the workspace:

GROOVY
stage('Test') {
  steps {
    sh '''
      python3 -m venv .venv
      . .venv/bin/activate
      pip install --upgrade pip
      pip install -r requirements.txt
      python -m pytest --junitxml=reports/junit.xml
    '''
  }
}

Two small details matter. The sh step uses /bin/sh by default, which may not understand source, so use the portable . .venv/bin/activate. And each sh step is a separate process, so the activation only lasts inside this one multi-line script; put all commands that need the environment in the same step, or call .venv/bin/python directly. Pinned requirements (a lock file or == versions) give reproducible builds just as lock files do in Node.

Caching dependencies

Downloading the same libraries on every build is the biggest waste of time in most pipelines. There are three layers to think about:

  • Persistent agents keep their local caches (~/.m2, ~/.npm, pip's cache) between builds for free. This is the simplest setup, with the downside that a polluted cache stays polluted.
  • Ephemeral agents (containers, cloud instances) start empty. Mount a persistent volume for the cache directory, or point the tools at an internal mirror.
  • A proxy repository such as an Artifactory or Nexus instance, or a registry mirror, caches downloads for the whole organization and also protects you when the public service is down.

Some plugins and tools restore and save caches keyed by a fingerprint of the dependency files. The idea is common across CI systems: compute a key from the lock file; if a cache with that key exists, restore it; if the lock file changed, the key changes and the build starts fresh and saves a new cache. Do not cache things that are hard to invalidate, such as the build output of your own project.

A stand-in cache key

This example computes a cache key the way such schemes usually do. It hashes the content of the dependency file together with the tool and OS name. The hash is a simple stand-in (not a cryptographic hash such as SHA-256), good enough to show the behavior: same lock file gives the same key, and any change gives a different one.

JS A stand-in dependency cache key

Press Run (or Ctrl + Enter) to see the console output here.

No output.

The first build of each lock file pays the download cost, later builds hit the cache, and returning to the older lock file finds the old cache again. A real cache also needs a size limit and cleanup rules, which this stand-in leaves out.

A second stand-in picks the commands a pipeline would run from the files it finds in a project. It is a toy model of the decisions above.

JS A stand-in project detector

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Whatever the language, keep the real build in a script or build file that works on a developer laptop. The Jenkinsfile should call it, not reimplement it. Then "it fails only in CI" becomes rare.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why is npm ci preferred over npm install in CI?

    npm ci gives reproducible installs from a clean state and rejects a lock file that does not match package.json.
  2. 2. What does the tools directive do in a Declarative Pipeline?

    The names refer to tool installations configured in Jenkins. Jenkins installs or locates them and updates the PATH.
  3. 3. Why use . .venv/bin/activate rather than source inside an sh step?

    The dot command is the portable form that works in sh, while source is a bash feature.
  4. 4. A good dependency cache key is usually based on:

    When the lock file changes, the key changes and the cache is rebuilt. Otherwise the cache can be reused.

Summary

  • Jenkins runs your build tools; choose versions with the tools directive, wrappers, containers or baked-in agent images, and pin them.
  • Use batch, non-interactive flags in CI (-B -ntp for Maven, --no-daemon --console=plain for Gradle).
  • Use npm ci and commit lock files; keep Python dependencies in a virtualenv with pinned versions.
  • Cache downloads with persistent agents, volumes or a proxy repository, and key caches by the lock file.
  • Keep the real build in a script that runs the same on a laptop.

Next, check that the code works and report the results in Tests, JUnit Reports and Quality Gates.