Skip to content
Tutorials Center
My learning

Topics

Lesson 5 of 38 ยท Tests and Assertions

Assertions and Failure Reports

Learn how pytest rewrites the plain assert statement to explain failures: comparing strings, lists, dicts and sets, custom messages, and the -vv option for full diffs.

In unittest you must remember a family of methods: assertEqual, assertIn, assertTrue, assertDictEqual, assertListEqual, and so on. In pytest there is one tool, the Python assert statement. That seems too simple to be useful, because a bare assert normally tells you only AssertionError and nothing about why.

pytest solves this with assertion rewriting. When it imports your test files, it quietly rewrites each assert so that, when it fails, the report includes the values of every sub-expression. You write plain Python, and you get detailed reports for free.

What a plain assert can express

Anything that evaluates to true or false works:

Python
assert result == 42
assert name != ""
assert "error" in message
assert user is not None
assert isinstance(value, int)
assert all(item > 0 for item in prices)
assert 0 < ratio <= 1

Because it is ordinary Python, you can combine conditions with and, or and not, call functions, and use comprehension expressions. Keep the expression readable, though; a complicated assert is hard to debug.

Reports for common types

pytest has special comparison reports for strings, lists, tuples, dicts, sets and dataclasses. See them side by side:

Python Failure reports for different types

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Look at what each report gives you:

  • Strings: a character-level diff with ? markers under the difference.
  • Lists: the first index where they differ, plus length differences.
  • Dicts: which keys differ, which are extra on either side.
  • Sets: items only in the left and only in the right.
  • in: the container's contents.

Compare that with writing assertDictEqual and reading a wall of repr text.

Sub-expressions are shown too

When an assert contains function calls, pytest shows what each returned:

Python Seeing intermediate values

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

In the report, lines starting with + where show the value returned by each call and the object whose attribute was read. This is the feature that lets you skip writing messages in most tests.

Adding your own message

Python lets you add a message after a comma. pytest shows it, and the introspection still appears:

Python
assert len(errors) == 0, f"expected no errors but got {errors}"

Use a message when the value alone does not explain the situation, for instance when you assert a boolean (assert is_valid(x) fails with just False). Do not repeat what the code already says.

Python Messages when the expression is not enough

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Seeing everything with -vv

By default pytest truncates long diffs so that the report stays short. Add -vv when you want the full picture:

Python Truncated and full diffs

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The first run summarizes the differing item. With -vv you get the whole diff. (If a report in a real run says "Use -v to get more diff", this is what it means.)

Assertions that look like they work but do not

Two classic traps:

A tuple is always true. The code below never fails, because it asserts a non-empty tuple:

Python
assert (x == 1, "x should be one")   # BUG: parentheses make a tuple
assert x == 1, "x should be one"     # correct

pytest warns about this exact mistake (assertion is always true), but make it a habit to put the comma outside.

Running with python -O removes asserts. Python's optimize flag strips every assert, so tests silently pass. Never run your test suite with -O, and never use assert in production code to validate user input; use raise for that.

When assertions are rewritten

Rewriting happens on import, for test modules, conftest.py files and plugins. A helper module that is not a test file is imported normally, so a failing assert inside it will not get a rich report. If you move assertions into a shared helper file and want them rewritten, register it:

Python
import pytest
pytest.register_assert_rewrite("tests.helpers")

Do this before the helper is imported, usually at the top of conftest.py.

A comparison table

You want to check Write Report shows
equality assert a == b both values and a diff
inequality assert a != b both values
containment assert x in items the container
identity assert a is None the value
type assert isinstance(a, int) the value and its type
truthiness assert items the object
all or any assert all(...) the generator expression result
floats assert x == pytest.approx(0.3) see approx
exceptions with pytest.raises(...) see raises

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Why does a plain assert give detailed failure output in pytest?

    pytest's assertion rewriting instruments each assert in test modules so a failure includes the evaluated sub-expressions and diffs.
  2. 2. What is wrong with: assert (total == 10, "total must be ten")?

    A non-empty tuple is truthy. Write assert total == 10, "message" with the comma outside any parentheses.
  3. 3. Which option shows the full, untruncated diff for long comparisons?

    Each -v raises the verbosity; -vv disables diff truncation.
  4. 4. When is an assertion message worth adding?

    Rewriting already shows values. A message helps when the expression result (like False) hides the context.
  5. 5. Why should a test suite never run with python -O?

    Optimized mode strips asserts entirely. Tests would pass vacuously.

Summary

  • Use plain assert; pytest rewrites it to show values, sub-expressions and diffs.
  • Strings, lists, dicts and sets get specialised comparison reports.
  • Add a message with assert cond, "text" when the expression alone is unclear.
  • -vv disables diff truncation.
  • Avoid assert (cond, "msg") and never run tests with python -O.
  • Register helper modules with pytest.register_assert_rewrite if you want rich reports from them.

Next: Testing Exceptions