Skip to content
Tutorials Center
My learning

Topics

Lesson 25 of 36 ยท Data-driven Testing

Generating test data and test matrices

Generate test data in Robot Framework with FOR loops, random values with seeds, the Faker library and combination matrices, plus runnable Python stand-ins.

Hand-written data goes stale. A test that signs up with [email protected] fails the second time if the system forbids duplicate emails. Tests that need many combinations (browser times operating system times language) become too long to write by hand. This lesson covers generating test data: unique values, random values you can reproduce, fake but realistic people, and combination matrices.

The Robot code is shown read-only. The runnable parts are Python stand-ins, because the same ideas are what the libraries use underneath.

Loops and ranges

Robot has a FOR loop with several forms. IN RANGE counts, IN walks a list, and IN ENUMERATE gives an index.

TEXT
*** Test Cases ***
Create Ten Users
    FOR    ${i}    IN RANGE    1    11
        Create User    user${i}@example.test
    END

Check Prices
    @{prices}=    Create List    4.99    10    0.5
    FOR    ${index}    ${price}    IN ENUMERATE    @{prices}
        Log    Item ${index} costs ${price}
    END

Use loops for setup (creating data) more than for the checks themselves; checks in a loop stop at the first failure, which is why templates exist.

Unique values

Unique emails or names avoid clashes between runs. The built-in DateTime library and Evaluate keyword help:

TEXT
${stamp}=    Get Current Date    result_format=%Y%m%d%H%M%S%f
${email}=    Set Variable    qa.${stamp}@example.test

Another common trick is ${{uuid.uuid4().hex[:8]}}, an inline Python expression (Robot 5 and later), after Library String or an Evaluate import of uuid. Always use a reserved domain like example.test so test emails can never reach a real person.

Random data, with a seed

Random values find cases you did not think of, but a random failure you cannot repeat is useless. The fix is a seed: print it in the log, and allow it to be set from the command line. The stand-in below shows that the same seed reproduces the same data.

Python Stand-in: reproducible random data

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

In a suite, you would write something like Log Using seed ${SEED} in the setup, and run robot --variable SEED:42 tests/ to replay a failing run.

Fake people with Faker

The FakerLibrary package (pip install robotframework-faker) wraps Python's Faker. It produces names, addresses, emails and so on:

TEXT
*** Settings ***
Library    FakerLibrary    locale=en_US

*** Test Cases ***
Register A New Customer
    ${name}=     FakerLibrary.Name
    ${email}=    FakerLibrary.Email
    Register Customer    ${name}    ${email}

You can pass a seed to get the same fake data again. Faker data is realistic, which helps with display tests (long names, accents) but remember it is random: do not assert against the exact name, assert that what you typed is what appears.

Boundary values

A systematic source of data is the boundary of every rule. For an age field that accepts 18 to 65:

Value Expected
17 rejected (just below)
18 accepted (lower edge)
19 accepted
64 accepted
65 accepted (upper edge)
66 rejected (just above)

Six rows catch the classic off-by-one mistakes (< instead of <=). Put them in a template. This is a great example of generated data: you can compute the boundaries from the rule.

Ways to get test data

Hand-written Clear but small.
Computed Boundaries and ranges.
Random with seed Surprising and replayable.
Faker Realistic names and emails.
1 / 2

Combination matrices

Say a checkout must work for 3 browsers, 2 languages and 2 payment methods. That is 3 x 2 x 2 = 12 combinations. Writing 12 rows by hand is error-prone; generating them is simple. Pairwise (all-pairs) testing goes further: it picks a smaller set of combinations that still covers every pair of values, often cutting hundreds of cases to a few dozen.

This stand-in builds the full matrix with itertools.product and then a greedy pairwise reduction, so you can see how much smaller the second list is.

Python Stand-in: full matrix and pairwise

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

With three small parameters the saving is modest (12 down to 6), but it grows quickly: ten parameters with three values each have 59,049 combinations, while a pairwise set needs only a few dozen. The generated rows then feed a template test, either through a Python helper or by writing them to a CSV for DataDriver.

Setup and cleanup of generated data

Whatever you generate, remove it. Create the data in [Setup] or Suite Setup and delete it in the matching teardown, so the system under test stays tidy and the next run starts clean. If the system makes cleanup hard, use a dedicated test tenant or database that is reset on a schedule.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why log the random seed?

    The same seed gives the same random sequence, so a failure can be reproduced.
  2. 2. For an age field accepting 18 to 65, which values are the key boundaries?

    The values just outside and exactly on each edge catch off-by-one errors.
  3. 3. What is pairwise testing?

    It trades the full matrix for a much smaller set that still covers all pairs.
  4. 4. Which email domain is safest for generated test addresses?

    Reserved domains such as example.test can never deliver mail to a real person.

Summary

  • FOR loops with IN RANGE, IN and IN ENUMERATE create data in setup.
  • Make values unique with timestamps or uuids; use reserved domains.
  • Random data needs a logged, settable seed to be replayable.
  • FakerLibrary gives realistic names and emails.
  • Boundary values and combination matrices produce systematic data; pairwise shrinks big matrices.

Next: Reports, logs and output.xml