Skip to content
Tutorials Center
My learning

Topics

Lesson 13 of 29 ยท TestNG advanced

Data Providers and Factories

Write data-driven tests with TestNG: @DataProvider, parallel providers, @Parameters, @Factory, and how they compare with JUnit parameterized tests.

A test often has the same shape for many inputs: call a method with a value, compare the answer. Copying the test ten times is noise, and looping inside one test hides which input failed. TestNG solves this with data providers: you write the test once, a second method supplies rows of arguments, and TestNG runs the test once per row and reports each run separately.

This lesson covers the three TestNG tools for data-driven work: @DataProvider for rows of arguments, @Parameters for values that come from testng.xml, and @Factory for creating many instances of a test class. It finishes with a comparison to JUnit's parameterized tests. If you have not seen TestNG's basic annotations yet, start with Your First TestNG Test.

Real versus stand-in. The TestNG code in the plain code blocks is real, but the site cannot run the TestNG library, so it is read-only. The TryIt blocks run small JavaScript stand-ins that copy the idea (a provider returns rows, the runner calls the test once per row). They are not TestNG's code, and details such as report formats differ.

Three ways to feed a test

@DataProvider Rows of arguments from a method.
@Parameters Single values from testng.xml.
@Factory Many instances of one test class.

Pick by where the data lives and how much of it there is.

1 / 2

A first @DataProvider

A data provider is a method annotated with @DataProvider that returns Object[][] (rows and columns) or an Iterator<Object[]>. The test method names the provider with dataProvider = "..." and receives one row per call, with the columns as parameters:

Java
import org.testng.annotations.DataProvider;
import org.testng.annotations.Test;
import static org.testng.Assert.assertEquals;

public class LeapYearTest {

    @DataProvider(name = "years")
    public Object[][] years() {
        return new Object[][] {
            {2024, true},
            {2023, false},
            {1900, false},
            {2000, true},
        };
    }

    @Test(dataProvider = "years")
    public void detectsLeapYears(int year, boolean expected) {
        assertEquals(Calendar.isLeapYear(year), expected);
    }
}

TestNG runs detectsLeapYears four times and lists four results in the report. The name is optional; without it, the provider's method name is the name. Types must be compatible: TestNG converts what it can, such as boxing 2024 to int, but passing a String where an int is expected is an error.

A provider method can live in another class if it is static and you point to it with dataProviderClass = SharedData.class. That is handy when several test classes need the same data.

Notice the argument order in assertEquals(actual, expected): TestNG puts the actual value first, which is the reverse of JUnit. More on that in TestNG Assertions and Groups.

Seeing the mechanism

The stand-in below keeps a map of providers and a list of tests. For each test it asks the provider for rows and calls the body once per row, with the row index in the label. One row has a wrong expectation on purpose, so you can see how a single failing row is reported.

JS A data-driven runner

Press Run (or Ctrl + Enter) to see the console output here.

No output.

The output counts six runs: five rows plus the plain test. Row 4 (2100) fails and the message names the row, which is the main reason to prefer providers over a for loop inside one test. A loop stops at the first failure and gives no index.

Provider tips

  • Make rows readable. Wrap rows in a builder or use constants. A row like {"[email protected]", true, 200} is easy to misread.
  • Receive the method. A provider can declare a java.lang.reflect.Method parameter to return different data for different test methods, and an ITestContext parameter to read the suite configuration.
  • Lazy providers. Return Iterator<Object[]> to produce rows on demand, for example when reading a large CSV file.
  • Readable names. By default reports show the method name plus the row's values. If you need friendlier names, implement the ITest interface or use a listener; see your TestNG version's documentation for the exact mechanism.
  • Do not hide logic. If computing the rows needs if statements and loops, the provider itself needs tests, so keep it simple.

Parallel data providers

Normally the rows of one provider run one after another. Setting parallel = true on the provider lets TestNG run the rows on several threads:

Java
@DataProvider(name = "urls", parallel = true)
public Object[][] urls() {
    return new Object[][] { {"/home"}, {"/cart"}, {"/profile"}, {"/help"} };
}

The number of threads comes from the suite's data-provider-thread-count attribute (the default is 10). Parallel rows are only safe if the test method keeps no shared mutable state: no static counters, no shared WebDriver, no shared list that tests append to. The race on shared state is demonstrated in Listeners, Retry and Parallel Execution.

@Parameters: values from testng.xml

A data provider answers "which inputs?" inside Java. Sometimes the value should come from outside the code, such as a browser name or base URL that changes per environment. @Parameters reads named values from testng.xml:

Java
public class PriceTest {
    @Parameters({"currency", "rate"})
    @Test
    public void convertsTenUnits(String currency, double rate) {
        assertTrue(10 * rate > 0, currency + " rate must be positive");
    }
}
XML
<suite name="Prices">
  <parameter name="currency" value="EUR"/>
  <parameter name="rate" value="0.9"/>
  <test name="Euro">
    <classes><class name="shop.PriceTest"/></classes>
  </test>
</suite>

TestNG converts the text from the XML to the method's parameter types. If a parameter might be missing, add @Optional("default") to the method parameter. @Parameters also works on configuration methods such as @BeforeClass. The XML side is covered in testng.xml, Suites and Test Configuration.

@Factory: many instances of one class

Normally TestNG creates one instance of a test class. A factory is a method annotated with @Factory that returns the instances to run, each built with different constructor arguments:

Java
public class PriceFactoryTest {
    private final String currency;
    private final double rate;

    public PriceFactoryTest(String currency, double rate) {
        this.currency = currency;
        this.rate = rate;
    }

    @Factory
    public static Object[] create() {
        return new Object[] {
            new PriceFactoryTest("EUR", 0.9),
            new PriceFactoryTest("GBP", 0.8),
            new PriceFactoryTest("JPY", 150),
        };
    }

    @Test public void convertsTenUnits() { /* ... */ }
    @Test public void rateIsBelowOne()   { assertTrue(rate < 1); }
}

Every test method runs on every instance, so each instance carries its own fields. That is the difference from a data provider: a provider varies the arguments of one method, a factory varies the state of the whole class. Use a factory when setup or several tests depend on the same configuration, for example "run this whole class against each of three browsers".

The stand-in shows both: a suite parameter feeding one instance, and a factory producing three instances. The JPY instance fails one test on purpose.

JS @Parameters and @Factory stand-ins

Press Run (or Ctrl + Enter) to see the console output here.

No output.

The last line prints 2,2,2: each instance counted its own two calls. With a single shared instance the count would be 6, which is why instance fields in a factory-built class are safe from cross-talk (and why fields in a normal TestNG class, shared by all its methods, are not).

TestNG vs JUnit for data-driven tests

Need TestNG JUnit (Jupiter)
Rows of arguments @DataProvider returning Object[][] @ParameterizedTest with @CsvSource, @MethodSource
Simple lists provider method @ValueSource, @EnumSource
Values from build/suite file @Parameters + XML system properties, @EnabledIf, config
Same tests, different class state @Factory @Nested classes or a @TestFactory
Parallel rows parallel = true on provider parallel execution through configuration

JUnit's approach is declarative: the sources sit right above the test and the conversion to types is built in. TestNG's approach is programmatic: the provider is plain Java, so it can read files, query a database, or build objects. For the JUnit side, see Parameterized and Dynamic Tests.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does a method annotated with @DataProvider usually return?

    Each inner array is one row of arguments. TestNG calls the test method once per row.
  2. 2. What is the main reason to use a data provider instead of a for loop inside one test?

    A loop stops at the first failure and gives no per-input result. A provider runs and reports every row separately.
  3. 3. Which annotation creates several instances of a test class, each with different constructor arguments?

    @Factory returns the instances to run. A data provider varies the arguments of one method; a factory varies the state of whole instances.
  4. 4. A provider is marked parallel = true. What must be true of the test method?

    Rows run on several threads at once. Shared mutable fields or objects cause races and flaky results.

Summary

  • @DataProvider returns rows of arguments; TestNG runs the test once per row and reports each run.
  • A provider can live in another class (dataProviderClass, static) and can be lazy (Iterator<Object[]>).
  • parallel = true runs rows on several threads; the test must be free of shared mutable state.
  • @Parameters injects values from testng.xml; use @Optional for defaults.
  • @Factory creates many instances of a test class, each with its own state.
  • JUnit expresses the same ideas declaratively with @ParameterizedTest and sources.

Next: testng.xml, Suites and Test Configuration.