Skip to content
Tutorials Center
My learning

Topics

Lesson 38 of 38 ยท Projects and Practice

Practice: Reading and Fixing Failing Tests

Learn to read pytest failure reports, use -x, -vv, -l and --tb options, and fix a project with three planted bugs. Hands-on exercises with runnable editors and a debugging checklist.

You will spend more time reading failing tests than writing passing ones, so reading them well is a core skill. A pytest failure report is dense but very regular. Once you know where to look, most bugs are visible within seconds, before you open a debugger.

This practice lesson has three parts: a guided tour of failure reports, a hunt for planted bugs in a small project (the editors are live, so fix the code and rerun), and a checklist plus the command-line options you will reach for every day.

Part 1: anatomy of a failure report

Run this file. Every test fails on purpose, each in a different way:

Python A tour of failure messages

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Take the first failure apart. The report for each test has the same shape:

TEXT
____ test_numbers ____              <- the test name
test_tour.py:4: in test_numbers    <- file, line, function (the traceback)
    assert 2 + 2 == 5              <- the failing line
E   assert (2 + 2) == 5            <- pytest shows the expression it checked

The lines starting with E are pytest's explanation. For collections it shows a diff: for the dictionaries it names exactly the differing key, for lists the first position that differs, and for long strings a character-level hint. Read the E lines first. They usually say everything.

Three details worth noticing in the output you just produced:

  • test_exception fails with KeyError: 'b' and the traceback shows the exact line. An unexpected exception counts as a failure (reported as an error in the test body), while an unexpected passing pytest.raises reports DID NOT RAISE.
  • test_float fails because 0.1 + 0.2 is 0.30000000000000004 in binary floating point, which the report does not make obvious. Compare floats with pytest.approx(0.3).
  • The summary at the bottom lists one line per failure, which you can paste into pytest path::name to rerun just that test.

Part 2: find three planted bugs

A tiny text-statistics module has three bugs, and the tests are correct. Change the implementation (not the tests) until all pass. The test report below the code is your only clue. Edit and run as often as you like.

Python Exercise: fix the module, not the tests

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Work through it like this: run it, read the E lines of the first failure, form a hypothesis, change one thing, rerun. Do not skim: each failure names a different bug.

Stuck? The three bugs are: (1) the regular expression only matches lower-case letters, but the input has capitals, so the text must be lowered first; (2) the sort key does not break ties alphabetically, so sort by (-count, word); (3) dividing by the number of words crashes for empty text. Fix them one at a time and watch the failure list shrink.

Part 3: debugging toolkit

When the E lines are not enough, use these options. Each one answers a specific question.

Option Question it answers
-x / --maxfail=N Stop at the first failure so you can focus on one
-l / --showlocals What were the local variables when it failed?
-vv Show full, untruncated diffs and values
--tb=short / --tb=line / --tb=no Control traceback size (long is the default, auto is smart)
-k name Run only tests matching a name (isolate the failing one)
--lf / --ff Rerun last failures only / failures first
-s Show print() output live (disables capturing)
-rA Summary of all outcomes with captured output
--pdb Drop into the debugger at the failure
--trace Break at the start of each test
-p no:randomly Remove random order while you isolate a problem

The next TryIt shows -l and -vv in action on one failure, and print output, which pytest captures and shows only for failing tests:

Python Locals and captured output in a failure

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Under the failing line you see every local variable (prices, discount, result) and the text that print wrote. The test expected 20 but the average (20) was halved by the discount: the test has the bug here. Always ask which side is wrong, the code or the expectation.

A debugging checklist

flowchart TB
    A["Test fails"] --> B["Read the E lines<br/>and the failing line"]
    B --> C{"Understand it?"}
    C -->|yes| D["Fix code or test,<br/>rerun that one test"]
    C -->|no| E["Add -l, -vv, print, -s"]
    E --> F{"Understand it now?"}
    F -->|yes| D
    F -->|no| G["Isolate: -k, -x, --pdb,<br/>smaller input"]
    G --> D
    D --> H["Run the whole suite"]

Read, hypothesize, change one thing, rerun the smallest scope, then widen.

  1. Read the message before touching anything.
  2. Reproduce in isolation: run just that test. If it passes alone, suspect shared state or order (hidden dependencies).
  3. Shrink the problem: reduce the input to the smallest failing case.
  4. Decide who is wrong: the code or the expectation. Check the specification, not your memory.
  5. Change one thing at a time, and rerun.
  6. Keep the test. When a bug is fixed, the test that caught it stays as a regression guard.
  7. Run the full suite, with random order if you use it, before you push.

More exercises

Try each on your own in the editor above (or your machine), and write the failing test first:

  1. Make most_common accept a min_length argument that ignores short words. Cover min_length=0 and min_length larger than every word.
  2. Add longest_word(text) that returns the first longest word, and None for no words.
  3. Write a test with pytest.raises for a function that should reject non-string input with TypeError, and make sure the message is checked with match=.
  4. Take any test of yours that uses time.sleep and make it instant by injecting the delay (flaky tests lesson).
  5. Run your project with --cov-branch and look at the Missing column (coverage lesson).

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. In a pytest failure report, what do the lines beginning with E contain?

    The E lines show the assertion's evaluated values, the diff of collections and the exception message. Read them first.
  2. 2. Which option shows the values of local variables at the point of failure?

    -l prints every local variable of each frame in the traceback, which often reveals the wrong value immediately.
  3. 3. A test passes when run alone but fails in the full suite. What should you suspect first?

    Another test probably leaves state behind. Use random order and fresh fixtures to isolate it.
  4. 4. Why is print output not shown for passing tests?

    Capturing keeps passing runs quiet. -s disables capture, and -rA shows captured output for all tests.

Summary

  • Failure reports have a fixed shape: test name, traceback, failing line, and E lines with pytest's explanation.
  • Collections produce diffs, exceptions show the line that raised, floats need pytest.approx.
  • Use -x, -k, --lf, -l, -vv, -s and --pdb to focus and inspect.
  • Change one thing at a time, ask whether the code or the test is wrong, and keep regression tests.
  • You have reached the end of this track's practice chapter: go back to the pytest track page to revisit any lesson.

Next: Back to the pytest track