Skip to content
Tutorials Center
My learning

Topics

Lesson 33 of 38 ยท CI and Best Practices

Continuous Integration with GitHub Actions

Run your pytest suite on every push with GitHub Actions: Python version matrix, dependency caching, coverage gates, JUnit reports, tox or nox, and a test that lints the workflow file itself.

Tests that only run on your laptop protect only your laptop. Continuous integration (CI) runs the whole suite automatically on every push and every pull request, on a clean machine, so "it works for me" stops being an excuse and a broken change cannot be merged unnoticed.

GitHub Actions is the most popular CI service for Python projects because it lives next to the code. The same ideas apply to GitLab CI, CircleCI or Jenkins: install, test, report, gate. This lesson shows a complete workflow, explains each part, and adds the pytest options that make CI output useful.

Workflow files run on GitHub's servers, so they cannot run in your browser. The YAML below is read-only, but the second runnable example shows a useful trick: a pytest test that checks the workflow file itself for common mistakes.

What a CI pipeline does

flowchart LR
    A["push or pull request"] --> B["fresh runner VM"]
    B --> C["check out code"]
    C --> D["install Python + dependencies"]
    D --> E["lint / type check"]
    E --> F["pytest with coverage"]
    F --> G{"all green?"}
    G -->|yes| H["merge allowed"]
    G -->|no| I["merge blocked"]

Each run starts from nothing, so missing dependencies and hidden local state show up immediately.

A complete workflow

Save this as .github/workflows/tests.yml:

YAML
name: tests

on:
  push:
    branches: [main]
  pull_request:

concurrency:
  group: tests-${{ github.ref }}
  cancel-in-progress: true

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        python-version: ["3.11", "3.12", "3.13"]
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-python@v5
        with:
          python-version: ${{ matrix.python-version }}
          cache: pip

      - name: Install
        run: |
          python -m pip install --upgrade pip
          python -m pip install -e ".[test]"

      - name: Run tests
        run: >
          python -m pytest
          --cov=shopkit --cov-branch --cov-report=term-missing --cov-report=xml
          --cov-fail-under=90
          --junitxml=reports/junit.xml
          -ra

      - name: Upload reports
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: reports-${{ matrix.python-version }}
          path: |
            reports/junit.xml
            coverage.xml

Reading it from the top:

  • on chooses the triggers: every push to main and every pull request.
  • concurrency cancels an older run of the same branch when you push again, saving minutes.
  • matrix runs the job once per Python version, so you notice a break on 3.13 before your users do. fail-fast: false lets the other versions finish so you see all failures.
  • setup-python with cache: pip reuses downloaded packages between runs.
  • pip install -e ".[test]" installs the project plus the test optional dependency group (pytest, pytest-cov and friends), keeping test tools out of production requirements.
  • pytest options: coverage with a gate (--cov-fail-under), a JUnit XML report that CI dashboards parse into a list of failed tests, and -ra for a short summary of everything except passes.
  • if: always() uploads the reports even when tests fail, which is when you need them most.

Mark the test job as a required status check in the repository's branch protection settings; otherwise the red cross is only advice.

CI-friendly pytest options

Option Why it helps in CI
-ra Summary of skipped, xfailed and failed tests at the end of the log
--strict-markers --strict-config Typos become errors, not silent no-ops
-p no:cacheprovider No .pytest_cache writes on read-only checkouts
--maxfail=5 Stop early on a catastrophic break
--durations=10 Prints the 10 slowest tests, so slowness is visible
--junitxml=path Machine-readable results
-n auto Parallel runs with pytest-xdist
--color=yes Coloured output in the Actions log
-W error Turn warnings into failures
-m "not integration" Select a subset of tests by marker

Splitting fast and slow tests

A good pipeline gives quick feedback first and runs heavy checks later:

YAML
jobs:
  unit:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with: { python-version: "3.13", cache: pip }
      - run: pip install -e ".[test]"
      - run: pytest -m "not integration" -n auto

  integration:
    needs: unit
    runs-on: ubuntu-latest
    services:
      postgres:
        image: postgres:16
        env:
          POSTGRES_PASSWORD: postgres
        ports: ["5432:5432"]
        options: >-
          --health-cmd pg_isready --health-interval 5s
          --health-timeout 5s --health-retries 10
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with: { python-version: "3.13", cache: pip }
      - run: pip install -e ".[test]"
      - run: pytest -m integration
        env:
          DATABASE_URL: postgresql://postgres:postgres@localhost:5432/postgres

needs: unit means the integration job only starts if the quick unit job passed. services starts a real PostgreSQL container next to the job, the setup that database tests use for engine-specific behaviour. Secrets (API keys) belong in repository secrets and are exposed as ${{ secrets.NAME }}; never commit them, and never let pull requests from forks read them.

tox and nox: one command for every environment

Running the suite under three Python versions locally is tedious. tox and nox create a virtual environment per version and run your commands in each, and CI can call them too, so laptop and server do exactly the same thing:

INI
# tox.ini
[tox]
envlist = py311, py312, py313, lint

[testenv]
extras = test
commands = pytest --cov=shopkit {posargs}

[testenv:lint]
skip_install = true
deps = ruff
commands = ruff check src tests

tox -e py312 -- -k cart runs one environment and passes -k cart to pytest. nox does the same with a Python file (noxfile.py) instead of an ini file; uv run pytest and a Makefile are lighter alternatives.

Test your workflow file

A YAML typo only shows up after you push. Since workflows are just data, you can test them with pytest: parse the file and assert the rules your team cares about. This example uses PyYAML on an in-memory workflow:

Python A pytest that lints a workflow file

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The comment about on points out a real YAML quirk: YAML 1.1 parsers read an unquoted on as the boolean True. Tests like these are cheap guard rails for configuration that otherwise fails only in CI.

Habits for healthy CI

  • Keep the suite fast. Developers stop waiting for a 40-minute pipeline. Use --durations, parallel runs and the unit/integration split.
  • Make failures reproducible. Print the random seed (pytest-randomly), pin dependency versions or use a lock file, and use the same command locally and in CI.
  • Fix or quarantine flaky tests quickly. A suite people re-run "until green" trains them to ignore red.
  • Run the CI config locally with the same tool (tox, nox, act) so you debug on your machine, not via twenty pushes.
  • Cache wisely. Cache downloads, not test results.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does a python-version matrix do?

    A matrix expands one job definition into several jobs, one per value, so you test every supported version.
  2. 2. Why use if: always() on the artifact upload step?

    By default a failed step skips later steps. always() makes the upload happen regardless, so reports exist when you need them.
  3. 3. What does needs: unit mean for the integration job?

    needs creates a dependency between jobs, which gives fast feedback before heavier jobs start.
  4. 4. What is the purpose of tox or nox in a CI setup?

    They automate virtual environments and commands, so developers and CI run identical steps.

Summary

  • CI runs the suite on a clean machine for every push and pull request; make the check required.
  • A workflow checks out code, sets up Python (with caching and a version matrix), installs the project and runs python -m pytest with coverage, a gate and a JUnit report.
  • Split fast unit tests from slow integration tests with markers and needs.
  • tox/nox give laptop and CI the same entry point; services provide real databases.
  • Configuration files are data: test them, and keep the pipeline fast and trustworthy.

Next: Faster Suites and Flaky Tests