Skip to content
Tutorials Center
My learning

Topics

Lesson 18 of 37 ยท Scenario Outlines and Data

Examples Tables in Depth: Choosing the Right Data

Design the data behind Scenario Outlines: multiple named Examples blocks, equivalence classes, boundary values, avoiding combinatorial explosions and keeping tables readable.

Knowing the syntax of Scenario Outline is the easy part. The craft is deciding which rows to put in the table. Too few and bugs slip through; too many and the suite is slow, noisy and unreadable. This lesson covers techniques that give the most coverage per row, and the Gherkin features that keep tables organized.

Several Examples blocks

An outline may have more than one Examples block. Each block can have its own name, description and tags, which turns one flat table into labeled groups:

GHERKIN
Scenario Outline: Password validation
  When the user chooses the password "<password>"
  Then the password is <verdict>

  @valid
  Examples: Accepted passwords
    | password       | verdict  |
    | Tr0ub4dor&3    | accepted |
    | correct horse 7 | accepted |

  @invalid
  Examples: Rejected passwords
    | password | verdict  |
    |          | rejected |
    | short1   | rejected |
    | alllowercase1 | rejected |

Benefits:

  • The group names read like documentation in reports.
  • Tags allow running only @invalid rows (-Dcucumber.filter.tags="@invalid").
  • Different blocks can hold different columns only if the outline uses them all; every placeholder must exist in every block's header, so groups with different needs are better as separate outlines.

Equivalence classes

Instead of testing every possible value, divide inputs into classes that the system treats identically, then test one representative per class. For a shipping rule with the threshold at 50:

Class Representative
Below the threshold 10
At the threshold 50
Above the threshold 500
Invalid (negative) -5
Invalid (zero) 0

Testing 11, 12 and 13 adds nothing if 10 passes; they live in the same class.

Boundary values

Bugs gather at the edges: off-by-one errors, > instead of >=. For each boundary test the value just below, on, and just above the edge:

GHERKIN
Examples: Around the free-shipping threshold
  | total | shipping |
  | 49    | 5        |
  | 50    | 0        |
  | 51    | 0        |

The two techniques combine: pick the boundaries between classes, plus one representative far inside each class. The stand-in below generates boundary rows from a rule's thresholds and checks them against an implementation. It contains an off-by-one bug (> instead of >=) so you can see the boundary row catch it. Fix the comparison and rerun.

Java Stand-in: boundary rows catch an off-by-one

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with IO.readln() or a Scanner), it gets the next line from this box.

Only the row with exactly 50 fails. A table of "typical" values (10, 75, 120) would have passed and shipped the bug.

The combinatorial explosion

Outlines tempt people into full cross products. A checkout with three payment methods, four shipping options, two customer types and five countries has 3 x 4 x 2 x 5 = 120 combinations. Running 120 browser scenarios to cover a rule that depends on only two of the four factors wastes hours.

Questions that cut a table down:

  1. Does this column actually influence the outcome? If not, remove it from the table and fix a default in the step.
  2. Are two columns independent? Test each separately instead of crossing them.
  3. Can the exhaustive part move to a unit test? Fast unit tests are the right place for 120 combinations; keep a few representative scenarios.
  4. Can we use pairwise (all-pairs) testing? Most defects are triggered by one or two factors, so covering every pair of values at least once finds most of them with far fewer rows.

The stand-in below counts the full product for several factors and builds a greedy all-pairs set. Greedy pairwise is not minimal, but it shows the order of magnitude you save.

Java Stand-in: full product versus all-pairs rows

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with IO.readln() or a Scanner), it gets the next line from this box.

You will usually see a reduction from 120 rows to about 20. Real tools (PICT, AllPairs) do better. Use pairwise only where factors truly interact and no business rule says otherwise; rules like "never ship drones to Brazil" still need their own scenario.

Keeping tables readable

  • Align columns and keep the table narrow enough to avoid horizontal scrolling.
  • Name columns for the domain (order total, not v1).
  • One result per row, ideally one or two result columns.
  • Put the rule first: the outline's title or description states the rule so the table is just evidence.
  • Group with named Examples blocks instead of one table with a type column.
  • Show the interesting values only; defaults belong in the Background or the step definitions.
  • Avoid computed expectations such as =total*0.1; write the final number so the table documents the rule independently of the implementation.

Anti-patterns

  • The spreadsheet. An outline with twelve columns and fifty rows reads like a database export. Split it.
  • The universal outline. Combining unrelated behaviors with a scenarioType column and conditional steps inside the glue.
  • Random data. Examples should be deterministic. If you want property-style testing, use a unit-test framework.
  • Duplicated rows that cover the same equivalence class.

A quick design recipe

  1. Write the rule as one sentence.
  2. List its factors (inputs that change the outcome).
  3. For each factor, list classes and boundary values.
  4. Pick rows: boundaries first, then one representative per class, then any known bug cases.
  5. If rows cross more than two factors, think pairwise or move the combinations to unit tests.
  6. Name the Examples blocks so the table explains itself.

With tables, doc strings and outlines in hand you can describe data richly; the following chapters show how to control when and how often scenarios run, using hooks and tags. For a complete overview of this course, visit the track page.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why should boundary values be tested?

    Comparison mistakes (> versus >=) only show at or next to the threshold.
  2. 2. A rule depends on two of four factors. What should you do?

    Remove columns that do not influence the outcome to avoid a pointless explosion of rows.
  3. 3. What does pairwise testing aim to cover?

    Most defects involve one or two factors, so covering all pairs gives good coverage with far fewer rows.
  4. 4. What can separate Examples blocks of one outline have that rows of one table cannot?

    Each Examples block can be named and tagged, which groups rows and allows running them selectively.

Summary

  • Several named and tagged Examples blocks organize an outline and allow selective runs.
  • Use equivalence classes and boundary values to get coverage from few rows.
  • Avoid full cross products; remove irrelevant columns, use unit tests for exhaustive checks, or use pairwise selection.
  • Keep tables narrow, domain-named, deterministic and free of computed expectations.
  • Design rows from the rule: factors, classes, boundaries, then a name for each block.

Next: Hooks: Before, After and Friends