Skip to content
Tutorials Center
My learning

Topics

Lesson 25 of 38 ยท Plugins and Coverage

Coverage with pytest-cov

Measure which lines and branches your tests execute with pytest-cov and coverage.py, read the missing-lines report, exclude code sensibly and enforce a minimum in CI.

"How much of my code do the tests actually run?" is a question you can answer with a number. Code coverage records which lines of your program were executed while the tests ran. Lines that never ran are, by definition, lines no test could have caught a bug in.

In the pytest world coverage is measured by coverage.py, and the pytest-cov plugin connects it to the pytest command line. This lesson shows how to measure, how to read the report, what the numbers do and do not mean, and how to turn coverage into a guard rail in CI.

Install and run

Terminal
pip install pytest-cov
pytest --cov=shop --cov-report=term-missing

--cov=shop names the package or module to measure (use your package name, not the tests folder). --cov-report=term-missing prints a table in the terminal including the line numbers that never ran. Without --cov nothing is measured.

Here is a small module and a test suite that skips some behaviour, run for real:

Python A coverage report with missing lines

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Read the table column by column: Stmts is the number of executable statements, Miss how many never ran, Cover the percentage, and Missing the line numbers to look at. Here line 7 (the VIP discount) and line 11 (the unused function) are untested. The report just told you which tests to write next.

Branch coverage

Line coverage has a blind spot. In if vip: total *= 0.8 a test that only ever passes vip=True runs every line, yet never checks the case where the if is false. Branch coverage tracks each decision's two outcomes. Enable it with --cov-branch (or branch = true in config). The report gains Branch and BrPart (partially covered branches) columns.

Python Line coverage versus branch coverage

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The first run says 100%, the second reveals that the "vip is false" path (3->5, meaning the jump from line 3 to line 5 never happened) was never taken. Branch coverage is the better default.

Excluding what should not count

Some code is not worth testing: platform-specific fallbacks, if __name__ == "__main__": blocks, abstract methods. Mark them with a comment, or configure patterns:

Python
def debug_dump(state):  # pragma: no cover
    print(state)
TOML
# pyproject.toml
[tool.coverage.run]
branch = true
source = ["shop"]
omit = ["*/migrations/*"]

[tool.coverage.report]
show_missing = true
fail_under = 90
exclude_also = [
    "if __name__ == .__main__.:",
    "raise NotImplementedError",
    "if TYPE_CHECKING:",
]

With source configured you can just run pytest --cov. Keep exclusions rare and visible; an "excluded" file that grows is a way of fooling yourself.

Enforcing a minimum

--cov-fail-under=90 makes the pytest run fail when total coverage drops below 90 percent, even if every test passes. That turns coverage into a gate in CI: nobody can quietly delete tests or add untested code.

Python fail-under in action

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The tests pass, yet pytest returns a non-zero exit code because coverage is under the threshold. On a build server, that non-zero code is what turns the check red.

Other reports

Option Output
--cov-report=term Table in the terminal
--cov-report=term-missing Table plus missing line numbers
--cov-report=html Browsable site in htmlcov/ with highlighted lines
--cov-report=xml coverage.xml, read by services such as Codecov
--cov-report=json Machine-readable data

Combine them: pytest --cov=shop --cov-report=term-missing --cov-report=html. The HTML report highlights uncovered lines in red and is the best way to explore a big project.

What coverage does not tell you

flowchart LR
    A["Line executed"] --> B{"Was anything<br/>asserted?"}
    B -->|no| C["Counted as covered,<br/>but nothing is checked"]
    B -->|yes| D["Behaviour is verified"]

Coverage measures execution, not correctness.

A test with no assertions still gives 100 percent coverage. High coverage shows you what is definitely untested; it can never prove the opposite. Treat it as a map of blind spots:

  • Chasing 100 percent leads to silly tests. 85 to 95 percent with meaningful assertions beats 100 percent without them.
  • Look at the missing lines, not only the percentage; a critical function at 0 percent matters more than a trivial getter.
  • Consider mutation testing (tools such as mutmut) to check that your assertions would actually notice a bug.
  • Coverage of parallel runs (pytest-xdist) is combined for you by pytest-cov.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does the Missing column in --cov-report=term-missing list?

    term-missing adds the line numbers (and ranges) that never ran, so you know exactly where to add tests.
  2. 2. Why enable branch coverage?

    An if statement can have all its lines covered while its false path was never taken. Branch coverage tracks both outcomes.
  3. 3. What does --cov-fail-under=90 do?

    It is a quality gate: the run exits with a non-zero code when the threshold is not reached.
  4. 4. A test calls a function and asserts nothing. What does coverage say about that function?

    Coverage records execution only. Assertions are what turn execution into verification.

Summary

  • pytest --cov=package --cov-report=term-missing measures and shows untested lines.
  • Add --cov-branch so that both outcomes of every decision count.
  • Exclude only what is genuinely untestable, via # pragma: no cover or config.
  • --cov-fail-under=N makes coverage a CI gate; HTML and XML reports help humans and services.
  • Coverage finds blind spots; it does not prove tests are good.

Next: Property-Based Testing with Hypothesis