Skip to content
Tutorials Center
My learning

Topics

Lesson 16 of 38 ยท Parametrization and Marks

Parametrized Tests

Learn @pytest.mark.parametrize to run one test function with many input and expected-output pairs, stack decorators for combinations, and read the generated test ids.

Suppose you are testing a function that converts a temperature. You want to check zero, a positive number, a negative number, a decimal, and the boiling point of water. The first instinct is to copy the test five times, or to put five asserts in one test. The first is repetitive; the second hides which case failed because the test stops at the first bad assert.

Parametrization is the cure. You write the test once and give pytest a table of inputs and expected results. pytest then creates one separate test per row, each with its own name and its own pass or fail result.

The basic form

@pytest.mark.parametrize takes the names of the arguments (a comma-separated string) and a list of value tuples:

Python Your first parametrized test

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

One function, four test items. Their ids, such as test_conversion[0-32], are built from the values. If one row fails, the other three still run and are reported, so you see the full picture from one run.

flowchart LR
    D["test_conversion<br/>(celsius, fahrenheit)"] --> A["[0-32]"]
    D --> B["[100-212]"]
    D --> C["[-40--40]"]
    D --> E["[37-98.6]"]

One test function expands into one test per row of data.

Seeing which row fails

Change one expected value and look at the report:

Python A failing row

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The summary says exactly test_palindrome[Racecar-True] failed. You do not have to guess which of many asserts tripped. Fix the code (lowercase the text first), rerun, and all rows pass.

Argument names and values

  • The names string may be "a, b" or "a,b"; you can also pass a list or tuple ["a", "b"].
  • Each row must have one value per name. For a single name, pass plain values: @pytest.mark.parametrize("n", [1, 2, 3]).
  • Values can be anything: numbers, strings, lists, dicts, objects, even functions or classes.
  • The argument names must be parameters of the test function (or fixtures it requests).
Python One name, rich values

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

For list or dict values pytest cannot build a readable id from the value itself, so it uses the argument name and an index (numbers0, numbers1, ...). The next lesson shows how to give nicer ids.

Stacking decorators: all combinations

Two parametrize decorators on one test multiply: pytest produces the cartesian product of the values. This is a compact way to test every combination:

Python Stacked parametrize

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

You get 3 x 2 = 6 tests. The ids are ordered from the decorator closest to the function outward ([10-1], [10-2], ...). Stacking is great for independent dimensions (browser x language), but the count grows fast, so keep an eye on it.

Parametrize versus a loop

Why not simply loop inside the test?

Python
def test_conversion():
    for c, f in [(0, 32), (100, 212)]:
        assert celsius_to_fahrenheit(c) == f
Loop in one test Parametrize
stops at the first failing row every row runs independently
one result for all rows one result per row
cannot select a single row pytest "test_x.py::test_conversion[0-32]"
hard to mark one case as xfail each row can be marked
fine for tiny setup cost setup (fixtures) runs per row

Parametrize wins for nearly every table of cases. One exception: if each row has a very expensive setup that could be shared, restructure with a wider-scoped fixture.

Parametrizing test classes

The decorator also works on a class. The data is passed to every method, which must accept the arguments:

Python Parametrize on a class

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Six tests result: each method once per value.

Good data tables

A parametrized table is only as good as the cases in it. Include:

  • the typical case,
  • boundaries (zero, one, the maximum, one above the maximum),
  • empty and invalid values,
  • a case that once caused a bug (add rows as you find bugs; this is a regression test).

Keep the test body tiny: a call and one assert. If you need if statements inside a parametrized test to deal with different rows, split it into two tests.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does @pytest.mark.parametrize("x, y", [(1, 2), (3, 4), (5, 6)]) create?

    pytest generates one test item per row, each reported separately.
  2. 2. You stack @parametrize("a", [1, 2, 3]) over @parametrize("b", [10, 20]). How many tests run?

    Stacked decorators multiply into the cartesian product: 3 x 2 = 6.
  3. 3. Why is parametrize usually better than a for loop inside a test?

    A loop stops at the first failure and reports one result; parametrize reports every case on its own.
  4. 4. How many names must each row of values contain?

    Each row supplies one value per argument name listed in the first argument.

Summary

  • @pytest.mark.parametrize("names", [rows]) runs one test function once per row.
  • Every row is a separate test with its own id, pass/fail result and rerun option.
  • Stack decorators to test all combinations; watch how fast the count grows.
  • Use parametrize instead of loops or copy-pasted tests, and put boundaries and past bugs in the table.
  • The decorator also works on classes.

Next: IDs, Indirect Parametrization and Parametrized Fixtures