Skip to content
Tutorials Center
My learning

Topics

Lesson 31 of 44 ยท Releases and Deployment

Infrastructure as Code in Pipelines

Run Terraform safely from CI: fmt, validate and plan on pull requests, apply on merge, state locking, plan comments and scheduled drift detection.

So far your pipelines have deployed applications. But an application needs somewhere to live: networks, servers, databases, buckets, DNS records. If you create those by clicking in a cloud console, nobody can review the change, nobody can repeat it, and nobody remembers who did it. Infrastructure as Code (IaC) fixes this by describing infrastructure in files that live in Git, next to your application code. A tool such as Terraform (or OpenTofu, its open-source fork) reads the files, compares them with what really exists, and works out what to create, change or destroy.

Once infrastructure is code, it can travel through a pipeline like any other code. This lesson uses Terraform as the running example because its workflow (fmt, validate, plan, apply) maps neatly onto CI. The same ideas apply to Pulumi, CloudFormation, Bicep and others. None of the real commands can run on this site, since there is no cloud account here, so the commands and YAML are read-only. The runnable stand-ins show the core of what a planner does and how a state lock behaves.

The pull request flow

The safest pattern splits the work in two. On a pull request, the pipeline only reads: it checks formatting, validates the files and produces a plan, a preview of exactly what would change. A human reviews the plan together with the code. On merge to the main branch, a second job runs apply and makes the plan real.

flowchart LR
    A["Pull request opened"] --> B["fmt + validate"]
    B --> C["plan (read-only)"]
    C --> D["Plan posted as comment"]
    D --> E["Review and approve"]
    E --> F["Merge to main"]
    F --> G["apply (changes real infrastructure)"]

Pull requests only preview changes; merging is what applies them.

Each Terraform command has a distinct job:

Command What it does Changes real infrastructure?
terraform fmt -check Fails if files are not in canonical style No
terraform init Downloads providers and connects to the state backend No
terraform validate Checks syntax and internal consistency No
terraform plan Compares code, state and reality; prints the proposed changes No
terraform apply Performs the changes Yes

validate does not contact your cloud provider, so it is fast and needs no credentials. plan does contact it (to refresh what exists), so it needs credentials, but only read access is required for a pure preview.

State and why it needs a lock

Terraform keeps a state file: a record of which real resources it manages and their last known settings. Without it, Terraform could not tell "this bucket was created by me" from "this bucket was created by someone else". In a team the state must live in a shared remote backend (an object-storage bucket, Terraform Cloud and similar), never on one laptop and never committed to Git, because it can contain secrets.

Two applies running at the same time against one state could corrupt it or fight over the same resources. Backends therefore support state locking: the first operation takes the lock, and a second one fails fast with a message that the state is locked. Your pipeline should add a second layer with a concurrency group, so that runs queue up instead of failing:

YAML
concurrency:
  group: terraform-production
  cancel-in-progress: false

Notice cancel-in-progress: false. For tests you usually cancel older runs; for an apply you never do, because killing a run halfway through can leave infrastructure partly changed. Let it finish and make the next one wait. The Jobs, Steps and Dependencies lesson explains how jobs and conditions fit together.

Stand-in: a state lock

This stand-in models a lock as a single owner slot. The first run takes it; a second run is refused until the first releases it. The run order is scripted, so the output is the same every time.

JS Two pipeline runs fight for one state lock

Press Run (or Ctrl + Enter) to see the console output here.

No output.

The stand-in shows why the lock is released at the end of a run and why a crashed run can leave a lock behind. Terraform has a force-unlock command for that rare case, but it should be used only after you are sure nothing is still running.

A pull request workflow

Here is a typical plan workflow. It formats, validates and plans, then saves the plan so the apply job can reuse exactly the same plan later if you wish.

YAML
name: Terraform plan

on:
  pull_request:
    paths:
      - "infra/**"

permissions:
  contents: read
  id-token: write        # for OIDC login to the cloud
  pull-requests: write   # to post the plan as a comment

jobs:
  plan:
    runs-on: ubuntu-latest
    defaults:
      run:
        working-directory: infra
    steps:
      - uses: actions/checkout@v4

      - uses: hashicorp/setup-terraform@v3
        with:
          terraform_version: 1.9.8

      - name: Format check
        run: terraform fmt -check -recursive

      - name: Init
        run: terraform init -input=false

      - name: Validate
        run: terraform validate

      - name: Plan
        run: terraform plan -input=false -no-color -out=tfplan | tee plan.txt

      - name: Post plan as a comment
        uses: actions/github-script@v7
        with:
          script: |
            const fs = require("fs");
            const plan = fs.readFileSync("infra/plan.txt", "utf8");
            const fence = "`".repeat(3);
            await github.rest.issues.createComment({
              owner: context.repo.owner,
              repo: context.repo.repo,
              issue_number: context.issue.number,
              body: fence + "\n" + plan.slice(-60000) + "\n" + fence,
            });

Several details matter:

  • paths: infra/** means the workflow only runs when infrastructure files change. See Triggers for path filters.
  • -input=false makes Terraform fail instead of waiting for a prompt that nobody can answer in CI. -no-color keeps the log readable.
  • Authentication uses OIDC (id-token: write), so the workflow exchanges a short-lived token for cloud credentials and no long-lived access key is stored as a secret. The Environments, Approvals and Deployments lesson covers this.
  • The comment is truncated (slice(-60000)) because GitHub limits comment size. Production versions often update one existing comment rather than adding a new one on every push.
  • A pull request from a fork gets no secrets and a read-only token, so this workflow works for branches in the same repository. See Securing Your Pipeline before changing that.

Apply on merge

The apply job runs only on the main branch and uses an environment so a reviewer can be required before anything real changes.

YAML
name: Terraform apply

on:
  push:
    branches: [main]
    paths:
      - "infra/**"

concurrency:
  group: terraform-production
  cancel-in-progress: false

permissions:
  contents: read
  id-token: write

jobs:
  apply:
    runs-on: ubuntu-latest
    environment: production
    defaults:
      run:
        working-directory: infra
    steps:
      - uses: actions/checkout@v4
      - uses: hashicorp/setup-terraform@v3
        with:
          terraform_version: 1.9.8
      - run: terraform init -input=false
      - run: terraform apply -input=false -auto-approve

-auto-approve skips Terraform's own confirmation prompt, which is acceptable here because the approval already happened in two places: the reviewed pull request and the protected production environment. Without those gates, -auto-approve would be reckless.

There is one subtlety. This job re-plans at apply time, so if the real world changed between the pull request and the merge, the applied changes might differ from the plan the reviewer saw. Stricter setups save the plan file from the pull request and apply that exact file (terraform apply tfplan); Terraform then refuses to apply if the state has changed in the meantime.

Treat the plan as the thing you review. Read it for lines that say a resource will be destroyed or replaced. A one-line code change, such as renaming a resource, can look harmless in the diff and still delete a database in the plan.

What a plan really is

At its heart a plan is a set difference. Terraform has a desired set of resources (your code) and a current set (state, refreshed from the provider). Anything desired but missing is created; anything present but not desired is destroyed; anything in both whose settings differ is updated.

Stand-in: a tiny planner

This stand-in computes a plan from two plain objects and prints it in the familiar +, ~, - style. The real Terraform also handles dependencies, computed values and replacement rules, which this sketch leaves out.

JS Diff desired and current resources into a plan

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Look at the - line: bucket.legacy exists in reality but not in code, so the plan wants to remove it. That is exactly what you want to catch in review. If someone created a resource by hand and forgot to add it to the code, the next plan will propose deleting it. This leads to the last topic.

Drift detection

Drift means the real infrastructure no longer matches the code, usually because someone changed something in the console, or another tool touched it. You can detect it automatically by running a plan on a schedule and failing when it is not empty. The -detailed-exitcode flag makes plan exit with 0 for "no changes", 1 for an error and 2 for "changes present".

YAML
name: Drift check

on:
  schedule:
    - cron: "0 6 * * 1-5"   # 06:00 UTC on weekdays
  workflow_dispatch:

permissions:
  contents: read
  id-token: write
  issues: write

jobs:
  drift:
    runs-on: ubuntu-latest
    defaults:
      run:
        working-directory: infra
    steps:
      - uses: actions/checkout@v4
      - uses: hashicorp/setup-terraform@v3
      - run: terraform init -input=false
      - name: Plan and fail on drift
        run: terraform plan -input=false -detailed-exitcode

A failed run turns the schedule into an alarm: the team gets notified and decides whether to update the code to match reality or to apply and revert the manual change. The key idea is the same as everywhere in CI: the exit code decides pass or fail, as described in What Is CI/CD?.

Several environments

Real systems have dev, staging and production, and they should differ only in size and settings, not in shape. Two common ways to keep one codebase:

  • Workspaces: one configuration, several state files, chosen with terraform workspace select staging. Quick, but it is easy to forget which workspace is active.
  • One folder per environment (or one variables file each) calling shared modules. More files, but the target is explicit in the path, so a pipeline job for envs/staging cannot touch production by accident.
YAML
jobs:
  plan:
    strategy:
      matrix:
        env: [staging, production]
    steps:
      - uses: actions/checkout@v4
      - run: terraform -chdir=envs/${{ matrix.env }} init
      - run: terraform -chdir=envs/${{ matrix.env }} plan -out=plan.tfplan

Give each environment its own state and credentials, and protect the production apply with an environment approval.

Terraform also has provisioners, which run scripts on a resource after it is created. HashiCorp calls them a last resort: they are not part of the plan, cannot be previewed and are hard to repeat. Prefer a machine image, cloud-init, or a configuration tool run as its own pipeline step.

Good habits

  • Keep a separate state and pipeline per environment (staging, production), so a mistake in one cannot touch the other.
  • Pin the Terraform version and provider versions, and commit the dependency lock file (.terraform.lock.hcl).
  • Give the plan job read-only credentials and the apply job a more powerful role that only the protected environment can assume.
  • Never print secrets. Mark sensitive variables as sensitive = true and remember that plan output goes into comments.
  • Review destroys and replacements with extra care, and consider policy checks (for example, a tool that fails the plan when a public bucket appears).

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Why does the pull request workflow run only plan and not apply?

    A plan is a read-only preview. Keeping apply for the merge means infrastructure only changes after the code and the plan were reviewed.
  2. 2. What does state locking prevent?

    The lock lets one operation hold the state at a time. A second operation fails fast instead of corrupting the state or fighting over resources.
  3. 3. Why should an apply workflow use cancel-in-progress: false in its concurrency group?

    Cancelling tests is harmless. Cancelling an apply midway can leave resources in a partial state, so queued runs wait for the current one to finish.
  4. 4. What does terraform plan -detailed-exitcode return when there are changes to apply?

    Exit code 0 means no changes, 1 means an error and 2 means changes are present. That makes it usable for scheduled drift detection.
  5. 5. In the stand-in planner, a resource exists in current but not in desired. What does the plan propose?

    Anything present in reality but missing from the code is not desired, so the plan removes it. This is why manually created resources are dangerous.

Summary

  • Infrastructure as Code keeps infrastructure in Git, so changes are reviewed, repeatable and traceable.
  • On pull requests run fmt -check, validate and plan, and post the plan for reviewers; on merge run apply, ideally behind a protected environment.
  • State lives in a shared remote backend and is protected by a lock; add a concurrency group with cancel-in-progress: false for applies.
  • A plan is the difference between desired and current resources: create, update or destroy. Read destroys and replacements carefully.
  • Use OIDC instead of stored cloud keys, and read-only credentials for the plan job.
  • A scheduled plan -detailed-exitcode detects drift between code and reality.

Next, you will learn how to keep all of these pipelines quick as they grow: Making Pipelines Fast.