Skip to content
Tutorials Center
My learning

Topics

Lesson 29 of 37 ยท Reporting and CI

Cucumber in CI Pipelines: Maven, Gradle, GitHub Actions

Run Cucumber in continuous integration: Maven and Gradle setup, GitHub Actions and GitLab examples, tag-based stages, headless browsers, archived reports and exit codes.

A BDD suite proves its worth when it runs on every change without anybody remembering to start it. That is the job of continuous integration (CI): a server checks out the code, builds it, runs the tests and reports a pass or fail. This lesson shows how to run Cucumber from Maven and Gradle, how to wire both into GitHub Actions (with notes for GitLab CI and Jenkins), how to split the suite into stages with tags, and how to keep the reports after the build machine is gone.

Neither a CI server nor Cucumber runs on this site, so the build files and pipelines below are read-only. The runnable stand-in is a small Java planner that picks the tag expression for each pipeline event and shows how a test exit code gates the build.

Maven

With the runner suite class from Selenium setup (named like RunCucumberTest) on the test classpath, Surefire finds and runs it:

Terminal
mvn -B test
mvn -B test -Dcucumber.filter.tags="@smoke and not @wip"
mvn -B test -Dcucumber.plugin="pretty,junit:target/cucumber/junit.xml,html:target/cucumber/report.html"
mvn -B verify -Dheadless=true -Dbase.url=https://staging.example.test

-B (batch mode) removes download progress bars from CI logs. Any -D option becomes a system property, so it can set Cucumber parameters (cucumber.*) and your own settings (headless, base.url) alike. A build fails when any scenario fails, because Surefire exits with a non-zero code; that is all the CI server needs.

With Gradle (Kotlin DSL), the JUnit Platform needs to be enabled and the properties forwarded:

Kotlin
tasks.test {
    useJUnitPlatform()
    systemProperty("cucumber.junit-platform.naming-strategy", "long")
    // forward -D options given to Gradle
    listOf("cucumber.filter.tags", "cucumber.plugin", "headless", "base.url").forEach { key ->
        System.getProperty(key)?.let { systemProperty(key, it) }
    }
    outputs.upToDateWhen { false }       // always rerun acceptance tests
    testLogging { events("failed", "skipped") }
}
Terminal
./gradlew test -Dcucumber.filter.tags="@smoke" -Dheadless=true

GitHub Actions

YAML
name: acceptance-tests
on:
  pull_request:
  push:
    branches: [main]
  schedule:
    - cron: "0 3 * * *"          # nightly

jobs:
  smoke:
    if: github.event_name == 'pull_request'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-java@v4
        with:
          distribution: temurin
          java-version: 21
          cache: maven
      - run: mvn -B test -Dcucumber.filter.tags="@smoke and not @wip" -Dheadless=true
      - uses: actions/upload-artifact@v4
        if: always()               # keep reports even when tests fail
        with:
          name: cucumber-reports
          path: target/cucumber/

  regression:
    if: github.event_name != 'pull_request'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-java@v4
        with: { distribution: temurin, java-version: 21, cache: maven }
      - run: mvn -B test -Dcucumber.filter.tags="not @wip and not @slow-nightly" -Dheadless=true
        env:
          API_TOKEN: ${{ secrets.TEST_API_TOKEN }}
      - uses: actions/upload-artifact@v4
        if: always()
        with: { name: cucumber-reports, path: target/cucumber/ }

Points worth noticing:

  • if: always() on the artifact step is essential; otherwise a failing test run uploads nothing, exactly when you need the report.
  • Ubuntu runners include Chrome, so Selenium Manager can start it headless. Without a display, always pass --headless=new.
  • Secrets come from the CI secret store as environment variables, never from feature files or the repository.
  • The cache: maven option keeps dependencies between runs.

GitLab CI is the same idea: a test job with script: mvn -B test, artifacts: when: always with paths: [target/cucumber/], and reports: junit: target/cucumber/junit.xml so merge requests show failures. Jenkins does it with a Pipeline stage that runs the same Maven command, then junit 'target/cucumber/junit.xml' and an HTML publisher. Because Cucumber runs inside a normal build tool, every CI system can run it.

Stages with tags

Running the whole suite on every commit is slow. Split by purpose:

flowchart LR
    A["Commit / pull request"] --> B["Unit tests<br/>seconds"]
    B --> C["@smoke scenarios<br/>minutes"]
    C --> D["Merge"]
    D --> E["Full regression<br/>@regression, in parallel"]
    E --> F["Nightly: slow and cross-browser"]

Quick feedback on every change; the expensive checks run less often.

Stage Tag expression When
Pull request @smoke and not @wip every push
After merge not @wip and not @nightly each merge to main
Nightly not @wip schedule
Per feature area @checkout only when files in that area change

Tags keep these definitions in one place (Tags); the pipeline only picks an expression. Combine with parallel execution to keep each stage short.

Practical advice

  • Same command locally and in CI. If developers cannot reproduce a failure with one command, they will not fix it.
  • Pin versions: JDK, browser images and Cucumber version, so a new browser release does not silently break the build.
  • Start the application under test as part of the job (a service container, Docker Compose or a built-in test server), and wait for a health endpoint before running scenarios.
  • Time limits: set a job timeout so a hung browser cannot block the queue.
  • Keep reports: JUnit XML for the CI dashboard, the HTML report and screenshots as downloadable artifacts (Reports).
  • Do not auto-retry blindly. A rerun that turns red into green hides flakiness (Flaky suites).

A pipeline planner stand-in

This small program decides which tags to run for a CI event and simulates the exit-code gate that makes a failing scenario fail the build.

Java Choosing tags per pipeline event (Java stand-in)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with IO.readln() or a Scanner), it gets the next line from this box.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why use if: always() on the artifact upload step in GitHub Actions?

    By default, steps after a failed step are skipped. always() ensures reports and screenshots are kept for failed runs.
  2. 2. How does a failing scenario fail a Maven build?

    The build tool turns failed tests into a non-zero exit code, which every CI system treats as failure.
  3. 3. Which approach gives fast feedback on pull requests?

    Tag-based stages keep the pull request pipeline short while still covering everything regularly.
  4. 4. Where should API tokens used by the tests come from in CI?

    Secrets belong in the CI secret store and are injected at run time.

Summary

  • Cucumber runs through Maven or Gradle, so any CI system can run it; failed scenarios yield a non-zero exit code.
  • Pass tags, plugins and settings with -D options; run headless and pin tool versions.
  • Split the suite into stages by tag: smoke on pull requests, regression after merge, slow checks nightly.
  • Always archive JUnit XML, HTML reports and screenshots, and keep secrets in the CI store.

Next: Declarative versus imperative scenarios