Skip to content
Tutorials Center
My learning

Topics

Lesson 87 of 149 ยท Beyond the Basics

Testing with unittest and doctest

Learn to test Python code with unittest and doctest. Write test cases, use assertions, check exceptions, share setup, and read test failures with confidence.

Every time you change a program, you risk breaking something that used to work. You could re-run the program by hand and eyeball the output, but that gets slow and unreliable as code grows. Automated tests are small pieces of code that call your functions with known inputs and check the results. They run in seconds and never forget a case.

Python ships with two testing tools in its standard library. unittest is a full framework for organizing and running tests. doctest checks examples written right inside your docstrings. In this lesson you will use both, learn the most useful assertions, test for errors, and read a failure report like a pro.

Why bother with tests?

Tests pay off in three ways:

  • They catch regressions. When a later change breaks old behavior, a test fails right away, while the change is still fresh in your mind.
  • They let you refactor without fear. You can rewrite a messy function, and if the tests still pass, it still does its job.
  • They document behavior. A test named test_empty_cart_costs_nothing tells the next reader exactly what the code promises.

The simplest test is an assert statement: assert add(2, 3) == 5. It works, but it stops at the first failure and gives a bare error. A test framework runs every test, reports which ones failed and shows what the values actually were.

Your first unittest test case

With unittest, you write a class that inherits from unittest.TestCase. Each method whose name starts with test is one test. Inside, you use assertion methods such as self.assertEqual():

Python A first test case

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The runner lists each test with ok, then a summary line: Ran 3 tests and OK. With verbosity=2 you see every test name; the default verbosity prints just a dot per passing test.

Name tests like sentences. test_collapses_extra_spaces explains itself in the report. test2 tells you nothing when it fails six months from now.

Running tests

On your own computer, you normally put tests in files named test_*.py and let unittest find them. There are two common ways:

Python
# At the bottom of a test file, so "python test_slugify.py" runs it
if __name__ == "__main__":
    unittest.main()
TEXT
# From the project folder: discover and run every test_*.py file
python -m unittest
python -m unittest -v          # verbose: one line per test

unittest.main() looks for test classes in the module that is running as the main program. It also reads the command line and ends the program with sys.exit() when the tests finish. In a notebook or in the editors on this site, call it as unittest.main(argv=[""], exit=False) to skip both. The examples on this page go one step further and hand the test class to the runner directly, which prints the report as normal output:

Python
suite = unittest.defaultTestLoader.loadTestsFromTestCase(TestSlugify)
unittest.TextTestRunner(stream=sys.stdout, verbosity=2).run(suite)

stream=sys.stdout simply prints the report as normal output; by default the runner writes to the error stream.

Reading a failure

Tests are most valuable when they fail, so it pays to read failures well. The function below has a bug. Run it and study the report:

Python When a test fails

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Each failure section tells you three things:

  1. Which test failed: test_average_of_three (TestAverage...).
  2. Where: the line in the test that made the failing assertion.
  3. Why: AssertionError: 5.0 != 4, meaning the first value (what your code returned) did not equal the second (what you expected).

The last line, FAILED (failures=2), summarizes the run. Remove the + 1 and run it again to see both tests pass.

unittest also separates failures (an assertion was false) from errors (the test crashed with some other exception, such as a TypeError). Both mean "fix me", but errors often point at a broken test rather than broken code.

The most useful assertions

TestCase has many assertion methods. These cover most needs:

Method Passes when
assertEqual(a, b) / assertNotEqual(a, b) a == b / a != b
assertTrue(x) / assertFalse(x) x is truthy / falsy
assertIs(a, b) / assertIsNone(x) a is b / x is None
assertIn(item, container) item in container
assertIsInstance(obj, cls) isinstance(obj, cls)
assertAlmostEqual(a, b) a and b are equal to 7 decimal places
assertGreater(a, b), assertLess(a, b) a > b, a < b
assertRaises(Error) the code inside raises Error

Prefer the specific method over assertTrue(a == b). When it fails, assertEqual shows both values (and a diff for lists, dicts and long strings), while assertTrue only says False is not true.

For floats, never compare with assertEqual: 0.1 + 0.2 is not exactly 0.3 (see Numbers). Use assertAlmostEqual.

Testing exceptions, setup and sub-tests

Good tests check the unhappy paths too. assertRaises works as a context manager: the test passes only if the code inside the with block raises the expected exception.

When several tests need the same starting objects, put them in a setUp method. It runs before each test, so every test gets a fresh copy and tests can't interfere with each other. (There is also tearDown, which runs after each test, for cleanup.)

Python assertRaises and setUp

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

assertRaisesRegex also checks that the error message matches a regular expression, which catches the case where the right exception type is raised for the wrong reason.

Many inputs with subTest

To check one rule against many inputs, loop over them inside self.subTest(). Each input is reported separately, and one failing case doesn't hide the others:

Python Table-driven tests with subTest

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Try breaking is_leap_year (for example, delete the or year % 400 == 0 part). The report names the exact year that failed.

doctest: examples that check themselves

Docstrings often include examples of how to call a function. doctest finds lines that look like an interactive Python session (starting with >>>), runs them, and checks that the output matches what's written below them:

Python Docstring examples as tests

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The expected output must match exactly, character for character, as the REPL would print it. That's why the results are written with quotes ('A.L.'): the REPL shows the repr() of a string. Change one expected value and run again to see doctest report the mismatch.

In a real file you would run every docstring in the module at once:

Python
if __name__ == "__main__":
    import doctest
    doctest.testmod()          # silent when everything passes; add verbose=True to see each check

Or from a terminal: python -m doctest my_module.py -v. testmod() checks every docstring in the module that is running, and it works in these editors too. run_docstring_examples(), used above, is handy when you want to check one function at a time.

unittest or doctest? Use doctest for short, illustrative examples that also serve as documentation. Use unittest for thorough checks: edge cases, error handling, many inputs and setup code. Many projects use both.

Tests in a real project

A typical small project keeps tests in their own folder:

TEXT
my_project/
  shop/
    __init__.py
    cart.py
  tests/
    __init__.py
    test_cart.py

Run python -m unittest from my_project/ and it discovers tests/test_cart.py automatically. Keep each test small, focused on one behavior, and independent of the others (they may run in any order).

Many Python developers also use pytest, a popular third-party framework installed with pip (see Packages and pip). It lets you write tests as plain functions with plain assert statements and still get detailed failure reports. It can run your existing unittest tests too, so what you learned here carries over.

A useful habit is test-driven development (TDD): write a failing test for the behavior you want, write just enough code to make it pass, then clean up the code while the test keeps you safe. Even if you don't follow it strictly, writing a test for every bug you fix stops that bug from ever coming back.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Which methods in a unittest.TestCase subclass does the runner treat as tests?

    The loader collects methods whose names begin with test. Other methods, such as helpers, are not run as tests.
  2. 2. When does setUp run?

    setUp runs before every test, giving each one fresh objects. tearDown is the one that runs after each test.
  3. 3. How do you check that parse("") raises a ValueError?

    The assertRaises context manager passes only if the code inside the with block raises the given exception.
  4. 4. Why is assertEqual(a, b) better than assertTrue(a == b)?

    assertEqual reports something like 5.0 != 4, while assertTrue can only say that False is not true.
  5. 5. What does doctest compare?

    doctest runs each >>> line and checks the result against the expected output text exactly, like a REPL session.

Summary

  • Automated tests catch regressions, make refactoring safe and document behavior.
  • In unittest, subclass TestCase and write methods starting with test; check results with assertEqual, assertIn, assertAlmostEqual and friends.
  • Use assertRaises (or assertRaisesRegex) for errors, setUp for shared fresh objects, and subTest for many inputs.
  • In scripts run tests with unittest.main() or python -m unittest; to run a single test case, load it and pass it to TextTestRunner.
  • doctest runs the >>> examples in docstrings and compares their output exactly.

Next: Virtual Environments and Project Setup, where you'll set up real Python projects on your own computer.