Skip to content
Tutorials Center
My learning

Topics

Lesson 26 of 36 ยท Testing and Publishing

Gradle Test Suites, Integration Tests and JaCoCo Coverage

Learn the JVM Test Suite plugin for integration tests, separate source sets, check lifecycle wiring, JaCoCo coverage reports and verification rules, and flaky-test strategies.

Unit tests are fast and live in src/test. Integration tests are slower, need a database or a running server, and deserve their own folder, dependencies and task. If you pile everything into test, the feedback loop slows down and you cannot run the quick tests alone.

This lesson shows how to add an integration test suite with Gradle's JVM Test Suite support, wire it into check, measure coverage with JaCoCo, and enforce a minimum. It continues from testing with JUnit.

Gradle cannot run on this website. Build code is read-only; Java stand-ins model coverage and test selection and run on Piston.

The jvm-test-suite plugin

The java plugin applies jvm-test-suite for you. The default suite is called test; you can declare more:

Kotlin
plugins {
    java
}

repositories { mavenCentral() }

testing {
    suites {
        val test by getting(JvmTestSuite::class) {
            useJUnitJupiter("5.11.4")
        }

        register<JvmTestSuite>("integrationTest") {
            useJUnitJupiter("5.11.4")

            dependencies {
                implementation(project())                    // main code of this project
                implementation("org.testcontainers:postgresql:1.20.4")
            }

            targets {
                all {
                    testTask.configure {
                        shouldRunAfter(test)                  // fast tests first
                    }
                }
            }
        }
    }
}

tasks.named("check") {
    dependsOn(testing.suites.named("integrationTest"))
}

What this creates:

  • a source set integrationTest with src/integrationTest/java and src/integrationTest/resources,
  • configurations integrationTestImplementation, integrationTestRuntimeOnly, ...,
  • a task integrationTest that compiles and runs them,
  • a line in check, so ./gradlew build also runs the slow tests, while ./gradlew test stays quick.

useJUnitJupiter("5.11.4") adds the right dependencies and calls useJUnitPlatform() for you, so you do not repeat the boilerplate from the previous lesson. implementation(project()) gives the suite access to the main classes of the same project.

Use shouldRunAfter rather than dependsOn for ordering preferences: it does not force one to run when the other is requested.

A typical command set

TEXT
./gradlew test                    # unit tests only (seconds)
./gradlew integrationTest         # slow tests only
./gradlew check                   # unit + integration + other verification
./gradlew build -x integrationTest   # skip a task for a quick local build

CI usually runs check (everything) while developers run test in their inner loop.

Test suites in tags

If you prefer one source set, JUnit tags give a similar split without extra folders:

Kotlin
tasks.test { useJUnitPlatform { excludeTags("integration") } }

tasks.register<Test>("integrationTest") {
    useJUnitPlatform { includeTags("integration") }
    testClassesDirs = sourceSets.test.get().output.classesDirs
    classpath = sourceSets.test.get().runtimeClasspath
}

Separate source sets are cleaner when integration tests need heavy dependencies that unit tests should not see.

Code coverage with JaCoCo

JaCoCo instruments bytecode while tests run and records which lines and branches executed. It is a core plugin:

Kotlin
plugins {
    java
    jacoco
}

jacoco {
    toolVersion = "0.8.12"
}

tasks.test {
    finalizedBy(tasks.jacocoTestReport)          // always make a report
}

tasks.jacocoTestReport {
    dependsOn(tasks.test)
    reports {
        xml.required = true      // for CI tools such as Sonar or Codecov
        html.required = true     // build/reports/jacoco/test/html/index.html
    }
}

tasks.jacocoTestCoverageVerification {
    violationRules {
        rule {
            limit {
                counter = "LINE"
                minimum = "0.80".toBigDecimal()
            }
        }
        rule {
            element = "CLASS"
            excludes = listOf("shop.generated.*")
            limit {
                counter = "BRANCH"
                minimum = "0.60".toBigDecimal()
            }
        }
    }
}

tasks.check {
    dependsOn(tasks.jacocoTestCoverageVerification)
}

Terms:

  • Line coverage: share of executable lines that ran.
  • Branch coverage: share of if/switch outcomes taken. It is stricter.
  • Verification rules: the build fails if a limit is not reached.

Coverage tells you what was not tested; it does not prove the tested code is correct. Do not chase 100%. A rule of 70 to 85% lines on business code, with generated code excluded, is a common compromise.

To merge unit and integration coverage, apply the jacoco-report-aggregation plugin or point executionData at several .exec files.

Stand-in: how coverage is counted

A coverage tool records which lines ran. This model "instruments" a small method by hand and reports line coverage.

Java Stand-in: line coverage counter (not JaCoCo)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with IO.readln() or a Scanner), it gets the next line from this box.

After test 1 only lines 1, 3 and 5 ran (60%), so the rule fails. Add the other two tests and coverage reaches 100%.

Splitting fast and slow work

Java Stand-in: unit tests first, integration on check

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with IO.readln() or a Scanner), it gets the next line from this box.

Flaky and slow tests

  • Isolate integration tests: containers started per suite, random ports, fresh temp directories.
  • Retry sparingly. The Test Retry plugin can rerun failures, but treat it as a bandage and fix the cause.
  • Run slow tests nightly if they take minutes, and keep check for pull requests as short as your team can tolerate.
  • Measure. Open the HTML report and sort by duration; a handful of tests often cause most of the time.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What is the benefit of a separate integrationTest suite?

    A separate suite isolates slow tests, lets them have heavy dependencies, and can be run independently of the unit tests.
  2. 2. How do you make ./gradlew build also run integrationTest?

    build depends on check, and check depends on verification tasks you attach to it.
  3. 3. Which JaCoCo setting fails the build when coverage is too low?

    Verification rules compare measured counters with minimums and fail the task, and therefore check, if a limit is not met.
  4. 4. Why is high coverage not a proof of correctness?

    A test that executes code without checking results still counts as coverage.

Summary

  • Declare extra suites in the testing { suites { ... } } block; each gets a source set, configurations and a task.
  • Wire slow suites into check; keep test fast.
  • JaCoCo records coverage; use verification rules to enforce minimums.
  • Prefer line and branch limits on business code, and exclude generated code.
  • Isolate and measure integration tests to avoid flakiness.

Next: Publishing to Maven repositories