Skip to content
Tutorials Center
My learning

Topics

Lesson 28 of 36 ยท Reporting and CI

CI with GitHub Actions and parallel runs

Run Robot Framework in continuous integration: a GitHub Actions workflow, exit codes, artifacts, secrets, Docker and parallel execution with pabot.

Tests that only run on one person's laptop do not protect a team. Continuous integration (CI) runs your suite automatically on every push or pull request, on a clean machine, and shows the result to everyone. Robot Framework fits CI well because it is a command-line tool with predictable exit codes and file output.

This lesson builds a GitHub Actions workflow, explains exit codes and artifacts, handles secrets, and introduces parallel execution with pabot. CI services and the robot command cannot run here, so the workflow files are read-only. The runnable examples are Python stand-ins for the exit-code rule and for splitting tests across workers.

What CI needs from your suite

  1. A single command that runs the tests (robot tests/).
  2. A non-zero exit code when something fails, so the CI job turns red.
  3. Files to keep: output.xml, log.html, report.html, screenshots.
  4. Headless browsers, since a CI machine has no screen.
  5. Pinned versions of Robot Framework and libraries, so a new release cannot break tomorrow's build.

For the last point, list dependencies in a requirements.txt:

TEXT
robotframework==7.1.1
robotframework-seleniumlibrary==6.7.0
robotframework-requests==0.9.7

Use versions you have actually tested; the numbers above are examples. Update them on purpose, in their own pull request.

Exit codes

Robot's exit code is the number of failed tests, up to 250. 0 means everything passed. Special values: 251 help or version printed, 252 invalid arguments, 253 run stopped by the user, 255 an unexpected internal error. CI only cares whether it is zero or not.

A CI run, step by step

Checkout Get the code.
Set up Python Pinned version.
Install pip install requirements.
robot Run the suite headless.

Then upload the result files, even when tests fail.

1 / 2

A GitHub Actions workflow

Save this as .github/workflows/robot.yml:

YAML
name: Robot tests
on:
  push:
    branches: [main]
  pull_request:

jobs:
  robot:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
          cache: pip
      - run: pip install -r requirements.txt
      - name: Run tests
        env:
          API_TOKEN: ${{ secrets.API_TOKEN }}
        run: >
          robot --variable BROWSER:headlesschrome
          --variable TOKEN:$API_TOKEN
          --outputdir results
          --exclude slow tests/
      - name: Upload results
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: robot-results
          path: results/

Points worth noticing:

  • if: always() uploads the results even when the tests failed, which is exactly when you need them.
  • The token comes from a repository secret and reaches Robot through a variable, never a file in the repository.
  • --exclude slow keeps pull-request runs fast; a scheduled workflow (on: schedule: with a cron line) can run the full suite at night.
  • GitHub-hosted Ubuntu runners include Chrome, so headlesschrome works with SeleniumLibrary.

Using Docker

For a repeatable environment, run the suite in a container that has the browsers and libraries baked in. A tiny Dockerfile:

TEXT
FROM python:3.12-slim
WORKDIR /work
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
CMD ["robot", "--outputdir", "results", "tests/"]

Mount the results folder with -v so the reports land on the host. This also lets you run the same container on any CI system.

Running in parallel with pabot

A thousand-test suite that takes two hours is a bottleneck. pabot (pip install robotframework-pabot) runs suites or tests in parallel processes and merges the results:

Terminal
pabot --processes 4 --outputdir results tests/
pabot --testlevelsplit --processes 8 tests/

By default pabot splits by suite file; --testlevelsplit splits down to single tests for better balance. At the end it produces one combined output.xml, log.html and report.html.

Parallel runs demand independent tests. Two tests that edit the same user, or rely on the order they run, will clash. Create unique data per test (see generated data), avoid shared files, and use pabot's #WAIT and resource locks (pabot library keywords such as Acquire Lock) only for the few things that cannot be shared.

Stand-in: exit codes

This stand-in applies the rule that CI uses: the exit code is the failed-test count, capped at 250.

Python Stand-in: exit code from results

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Stand-in: splitting work across workers

pabot's job is to balance work across processes. This stand-in assigns suites with known durations to 3 workers with a simple greedy rule (longest first, to the least busy worker) and shows the time saved.

Python Stand-in: balance suites across workers

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Do not let CI retry failing tests silently until they pass. Report retries, and fix flaky tests; otherwise the suite slowly stops meaning anything.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why use if: always() on the artifact upload step?

    By default later steps are skipped after a failure, and failed runs are when you need the reports.
  2. 2. What exit code does Robot return when all tests pass?

    Zero means success; otherwise the code is the number of failed tests (up to 250).
  3. 3. What must be true of tests before you run them in parallel with pabot?

    Shared data or order dependencies cause clashes when processes run at the same time.
  4. 4. Where should an API token for CI live?

    Secrets belong in the CI secret store and reach the run as variables or environment values.

Summary

  • CI needs one command, a meaningful exit code, headless browsers, pinned versions and saved artifacts.
  • A GitHub Actions workflow installs requirements, runs robot, and uploads results/ with if: always().
  • Secrets come from the CI secret store.
  • Docker gives a repeatable environment.
  • pabot runs suites or tests in parallel; tests must be independent.

Next: Writing custom Python libraries