Skip to content
Tutorials Center
My learning

Topics

Lesson 27 of 36 ยท Testing and Policy

Integration Testing and Ephemeral Stacks for Pulumi

Test real infrastructure with Pulumi: ephemeral stacks per test run, the Automation API, polling and retries, guaranteed cleanup, cost control and when integration tests are worth it.

Unit tests with mocks prove your logic. They cannot prove that the real cloud accepts your configuration, that the security group lets the load balancer reach the instances or that a Lambda actually answers. Integration tests deploy the program to a real (usually temporary) stack, check behavior from the outside and destroy the stack afterwards.

Real deployments are not possible on this site, so the TypeScript and shell commands are read-only. The runnable stand-ins show the parts you must get right in any harness: unique stack names, polling with timeouts, and cleanup that always runs.

The testing pyramid for infrastructure

flowchart TB
  A["Few: end-to-end tests of a full environment (minutes to hours)"] --> B["Some: integration tests per component or stack (minutes)"]
  B --> C["Many: unit tests with mocks, policy checks, previews (seconds)"]

Run the cheap layers on every commit and the expensive ones on a schedule or before releases.

Ephemeral stacks

An ephemeral stack is created for one test run and destroyed at the end. Give it a unique name (for example test-<short-sha>-<random>) so parallel CI jobs never collide, and so a crashed run leaves something you can identify and clean up.

Terminal
pulumi stack init test-a1b2c3 --non-interactive
pulumi config set aws:region eu-central-1 --stack test-a1b2c3
pulumi up --yes --stack test-a1b2c3
./smoke-test.sh "$(pulumi stack output url --stack test-a1b2c3)"
pulumi destroy --yes --stack test-a1b2c3
pulumi stack rm test-a1b2c3 --yes

If the smoke test fails, the shell must still run the destroy. Use trap in Bash or finally in code so cleanup always runs.

The Automation API as a test harness

The Automation API drives Pulumi from code, so a test can create a stack, deploy an inline program, read outputs and destroy it in one script. In TypeScript:

TypeScript
import { LocalWorkspace } from "@pulumi/pulumi/automation";
import * as aws from "@pulumi/aws";

const program = async () => {
  const bucket = new aws.s3.BucketV2("test-bucket", { forceDestroy: true });
  return { bucketName: bucket.bucket };
};

const stack = await LocalWorkspace.createStack({
  stackName: `test-${Date.now().toString(36)}`,
  projectName: "bucket-tests",
  program,
});

try {
  await stack.setConfig("aws:region", { value: "eu-central-1" });
  const up = await stack.up({ onOutput: console.log });
  const bucketName = up.outputs.bucketName.value;
  // ... call the AWS API or HTTP endpoint and assert
} finally {
  await stack.destroy({ onOutput: console.log });
  await stack.workspace.removeStack(stack.name);
}

Details worth knowing:

  • Use forceDestroy: true (or equivalent) on buckets and similar resources in test stacks, so destroy does not fail on non-empty resources.
  • Pulumi ships a Go integration test package (github.com/pulumi/pulumi/pkg/v3/testing/integration) that runs pulumi as a subprocess with ProgramTest options, if you prefer Go tests.
  • The LocalWorkspace needs the Pulumi CLI installed and a backend login, so CI needs credentials and an access token or a state bucket.

What integration tests assert

Check behavior, not resources:

  • An HTTP endpoint answers 200 with expected content.
  • A queue consumer processes a message sent by the test.
  • A database accepts a connection from the application role, and does not accept one from outside.
  • A DNS name resolves, a certificate is valid.

Many things are eventually consistent (DNS, load balancers, IAM). Tests must poll with a timeout and backoff instead of sleeping for a fixed time.

Node.js Stand-in: polling until a resource is ready, with a timeout

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

The exact attempt count varies with timing, but the shape is the point: poll, back off and give up after a deadline with a clear message.

Always clean up

A leaked test stack costs money and clutters the account. The harness must destroy resources even when assertions throw or the process is interrupted.

Node.js Stand-in: a test harness that always destroys the stack

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

Both runs leave 0 resources, even though the second test threw. For crashed CI jobs (killed runner), add a janitor: a scheduled job that lists stacks whose names start with test- and are older than a few hours and destroys them (pulumi stack ls --json gives the last update time).

Cost, speed and isolation

  • Use the cheapest equivalent. Test with a t3.nano, one availability zone and no NAT gateway if the test does not care.
  • Run in a dedicated account or subscription with budgets and alarms, never in production.
  • Parallelize by stack, not by sharing: ten tests can use ten stacks at once.
  • Tag everything with the stack name and a ttl tag so cleanup tools can find strays.
  • Limit what runs on pull requests. A full environment test on every PR is expensive; run it nightly and run component tests on PRs.

When integration tests are worth it

Choose them for components shared by many teams, risky changes (networking, IAM, databases), upgrades of provider major versions and disaster recovery exercises. For a simple stack that is only a handful of buckets, unit tests plus pulumi preview in CI usually give enough confidence.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why give an ephemeral test stack a unique name?

    Unique names prevent two CI jobs from sharing one stack and make it easy to find and clean stragglers.
  2. 2. What is the correct way to wait for an eventually consistent resource?

    Polling with a timeout adapts to fast and slow cases and fails clearly when something is really wrong.
  3. 3. Where should destroy be called in a test harness?

    Cleanup must run even if assertions fail, otherwise resources leak and cost money.
  4. 4. Which is a good reason to use forceDestroy on a bucket in a test stack?

    forceDestroy lets Pulumi delete a non-empty bucket, which keeps teardown reliable.

Summary

  • Integration tests deploy to real temporary (ephemeral) stacks and check behavior from the outside.
  • The Automation API turns create, up, outputs and destroy into code you can put inside a test.
  • Use unique stack names, poll with timeouts and clean up in finally; add a janitor for crashed runs.
  • Keep costs low with small instances, a dedicated account, tags and a sensible run schedule.
  • Use cheap unit and policy tests for every commit and reserve integration tests for risky or shared components.

Next: Policy as Code with CrossGuard