Skip to content
Tutorials Center
My learning

Topics

Lesson 55 of 73 ยท Quality and performance

Integration Tests and CI

Run Flutter integration tests on real devices with integration_test, learn what Patrol adds, use device farms, and automate tests and coverage in GitHub Actions.

Widget tests are fast, but they run in a fake environment. They cannot tell you that the real app starts, that the login flow works with the real keyboard, or that the platform plugin you added does not crash on Android 14. For that you need integration tests: the full app, running on an emulator, simulator or physical phone, driven by a script.

Then you want those tests to run on every pull request without anyone remembering to. That is continuous integration (CI). This lesson covers the integration_test package, what Patrol adds for native dialogs, device farms, and a GitHub Actions workflow with coverage.

Real vs stand-in. The Dart, YAML and shell commands are real but read-only: the site cannot start a Flutter app or a CI server. The live examples are JavaScript stand-ins for a scripted user flow with waits and for the arithmetic of a coverage gate.

Where integration tests fit

Widget tests Seconds. Fake environment.
Integration tests Minutes. Real app, real device.
Manual QA Hours. Exploration and feel.

Each level is slower and more realistic. Use all three.

1 / 2

The integration_test package

integration_test is part of the Flutter SDK. You add it as a dev dependency and put tests in an integration_test/ folder at the project root.

YAML
# pubspec.yaml
dev_dependencies:
  flutter_test:
    sdk: flutter
  integration_test:
    sdk: flutter

A test looks like a widget test, with one extra line: it starts the real main() of your app.

DART
// integration_test/login_flow_test.dart
import 'package:flutter_test/flutter_test.dart';
import 'package:integration_test/integration_test.dart';
import 'package:my_app/main.dart' as app;

void main() {
  IntegrationTestWidgetsFlutterBinding.ensureInitialized();

  testWidgets('a user can sign in and see the home page', (tester) async {
    app.main();
    await tester.pumpAndSettle();

    await tester.enterText(find.byKey(const Key('email')), '[email protected]');
    await tester.enterText(find.byKey(const Key('password')), 'correct-horse');
    await tester.tap(find.byKey(const Key('sign-in')));
    await tester.pumpAndSettle();

    expect(find.text('Welcome back, Ada'), findsOneWidget);
  });
}

The finders, tap, enterText and pumpAndSettle are the ones you already know. The differences from a widget test are important:

  • The app is real: real plugins, real rendering, real timing. Nothing is faked unless you fake it.
  • Frames run on the real clock, so animations take real time. pumpAndSettle waits for them.
  • You need a target. Start an emulator or plug in a phone, then run:
Terminal
flutter test integration_test/login_flow_test.dart -d <device-id>
flutter devices          # lists the ids

On Flutter web you run through flutter drive with a browser driver. Running integration tests on desktop works with -d windows, -d macos or -d linux.

Keep them reliable

Integration tests have a reputation for being flaky. Most flakiness has three causes, all fixable:

  1. Timing. Never use a fixed Future.delayed. Wait for a condition: pumpAndSettle, or a small helper that pumps until a finder matches or a timeout passes.
  2. Shared state. A previous run left a logged-in user on the device. Reset storage at the start of each test, or use a fresh user per test.
  3. The network. A real server can be down or slow. Point the app at a test backend, or inject a fake API for most flows (see the dependency idea), and keep only a few true end-to-end checks.

Keep the number of integration tests small and focused on critical journeys: sign up, sign in, the main purchase or save flow. Everything else belongs in faster tests.

Stand-in: a flow with a patient wait

The stand-in runs a scripted flow against a fake app whose screens appear after a delay. A helper waitFor polls until a condition holds, which is what a robust integration test does instead of sleeping for a guessed time.

JS Scripted flow with waitFor instead of sleeps

Press Run (or Ctrl + Enter) to see the console output here.

No output.

A fixed sleep(800) would pass today and fail when the server gets slower. waitFor passes as soon as the condition holds and fails with a clear timeout message when it never does.

Patrol: when you must leave the app

Plain integration_test can only see Flutter widgets. It cannot tap the Android "Allow location?" permission dialog, pull down the notification shade, toggle Wi-Fi or press the home button, because those are native, not widgets. The Patrol package extends the same idea with native interaction:

DART
patrolTest('asks for location permission and shows the map', ($) async {
  await $.pumpWidgetAndSettle(const MyApp());

  await $('Find me').tap();
  if (await $.native.isPermissionDialogVisible()) {
    await $.native.grantPermissionWhenInUse();
  }

  await $('Your location').waitUntilVisible();
});

Patrol adds a concise $('text') finder syntax, waitUntilVisible, and a $.native API for permission dialogs, notifications and system settings. It needs a small native setup and its own patrol test command. Reach for it when a flow involves permissions or notifications; skip it when your tests stay inside the app.

Device farms

Your laptop's emulator is one device. Users have thousands: different screen sizes, OS versions and manufacturers' quirks. A device farm is a cloud service that runs your tests on many real or virtual devices and returns logs, screenshots and videos. Firebase Test Lab, AWS Device Farm and BrowserStack are common options. Typical workflow:

  1. Build a test APK (Android) or a test bundle (iOS).
  2. Upload it with the service's command line tool or a CI step.
  3. Choose a device matrix, for example three Android versions and two screen sizes.
  4. Read the report and fail the build on any red result.

Running a full matrix on every commit is slow and costs money. A sensible compromise: one emulator on every pull request, the wide device matrix nightly or before a release.

Continuous integration with GitHub Actions

CI runs your checks on a clean machine every time code is pushed. A workflow file in .github/workflows/ describes the steps. This one analyzes, runs unit and widget tests with coverage, and uploads the report.

YAML
# .github/workflows/flutter-ci.yml
name: Flutter CI

on:
  pull_request:
  push:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: subosito/flutter-action@v2
        with:
          channel: stable
          cache: true            # caches the SDK and pub packages

      - run: flutter pub get
      - run: dart format --output=none --set-exit-if-changed .
      - run: flutter analyze
      - run: flutter test --coverage

      - uses: actions/upload-artifact@v4
        with:
          name: coverage
          path: coverage/lcov.info

Read it from top to bottom: on says when it runs; the job gets a fresh Ubuntu machine; the steps check out the code, install Flutter (with caching so it is quick), fetch packages, fail on unformatted code, run the static analyzer, and run tests. A failing step fails the pull request.

Integration tests need a device, so they go in a separate job. On Linux runners, an Android emulator needs hardware acceleration, and a popular approach is the reactivecircus/android-emulator-runner action. On macOS runners you can start an iOS simulator. These jobs take much longer, so run them on merges to main or on a schedule, not necessarily on every push.

YAML
  integration:
    runs-on: macos-latest
    needs: test
    steps:
      - uses: actions/checkout@v4
      - uses: subosito/flutter-action@v2
        with:
          channel: stable
      - run: open -a Simulator
      - run: flutter test integration_test

Release automation (building, signing and shipping) is the topic of CI/CD and releases.

Coverage

flutter test --coverage writes coverage/lcov.info, listing for every file which lines ran. Tools turn it into an HTML report (genhtml) or send it to a service such as Codecov. A line is covered if some test executed it.

Coverage tells you what is not tested; it does not prove that what is tested is correct. A test that runs a line but asserts nothing still counts. Use it to spot untested areas, and set a modest gate (for example "don't drop below 70%") rather than chasing 100%.

JS A coverage gate in a few lines

Press Run (or Ctrl + Enter) to see the console output here.

No output.

The total is a weighted average, so a big, poorly covered file pulls it down more than a small one. Listing the weakest files tells you where the next test pays off most.

A practical testing strategy

  • Put most logic into plain classes and test it with unit tests.
  • Widget-test each reusable component and each screen's states: loading, empty, error, data.
  • Write integration tests only for the few journeys that must never break.
  • Run format, analyze and tests on every pull request; run the heavy device matrix before releases.
  • Treat a flaky test as a bug: fix it or delete it, because a test nobody trusts is worse than none.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What can a plain integration_test NOT do on its own?

    integration_test sees Flutter widgets only. Native dialogs need something like Patrol.
  2. 2. Why use a waitFor helper instead of a fixed sleep in an integration test?

    A fixed sleep is either too short (flaky) or too long (slow). Waiting for a condition is neither.
  3. 3. Which command produces coverage/lcov.info?

    flutter test --coverage records which lines ran and writes the lcov file.
  4. 4. What is a sensible way to use a device farm?

    A wide matrix is slow and costs money, so use it on a schedule or before a release.
  5. 5. What does 100% line coverage prove?

    Coverage measures execution, not correctness. A test without assertions still adds coverage.

Summary

  • Integration tests run the real app on a device or emulator with integration_test and flutter test integration_test -d <id>.
  • Make them reliable by waiting for conditions, resetting state and limiting network dependence; keep them few and focused on critical journeys.
  • Patrol adds native interaction (permission dialogs, notifications) to the same style of test.
  • Device farms run your tests across many devices; use wide matrices sparingly.
  • A GitHub Actions workflow can format-check, analyze and test every pull request, collect coverage and run heavier jobs separately.
  • Coverage shows untested code, not test quality.

Next: Performance and DevTools