Skip to content
Tutorials Center
My learning

Topics

Lesson 8 of 36 ยท Concepts and Templates

Packer Build Lifecycle, Artifacts and the Manifest

Follow a Packer build from start to cleanup: machine creation, communicator, provisioners, artifact creation, post-processor chains, the manifest file and what happens on errors.

To debug a Packer build you must know what it does and in which order. A build is not just "run my script": Packer creates a machine, waits for it to be reachable, uploads files, runs your steps, shuts the machine down, snapshots the disk and then transforms the artifact. This lesson walks through that lifecycle, explains artifacts (the thing a build produces), and shows how to record them with the manifest post-processor so that later pipeline stages can find the image.

Packer cannot run on this site; HCL and JSON files are read-only. The Python stand-in models the lifecycle in order. Real output differs by builder and version, so treat our descriptions as the order of events, not as sample logs.

The lifecycle, step by step

For every source in a build, Packer performs these phases:

  1. Prepare: parse arguments, check types, and validate the configuration of each component. Errors here appear before anything is created.
  2. Create the machine: the builder uses the platform API. For amazon-ebs that means creating a temporary key pair and security group and launching an instance; for docker it starts a container; for qemu it boots a VM from an ISO.
  3. Connect: the communicator (SSH, WinRM or the platform's own channel such as docker exec) waits until it can reach the machine. Packer retries until ssh_timeout or the builder's equivalent expires.
  4. Provision: provisioners run in file order, each one fully finishing before the next begins.
  5. Stop and capture: the machine is shut down (gracefully if possible) and the builder creates the artifact (snapshot, AMI, exported disk file, committed image).
  6. Post-process: post-processors transform or publish the artifact.
  7. Cleanup: the temporary machine, key pair, security group and other build-time resources are deleted.
flowchart LR
  P["Prepare"] --> C["Create machine"] --> K["Connect"] --> V["Provision"] --> S["Stop + capture<br/>artifact"] --> PP["Post-process"] --> X["Cleanup"]
  V -. failure .-> E["Error handling<br/>(-on-error)"]
  E --> X

The order of events for one source.

Communicators

A communicator is how Packer talks to the machine:

Communicator Used for
ssh Linux and BSD images (most cloud builders default to it)
winrm Windows images
docker Docker builder (uses docker exec and docker cp)
none Builders that need no connection, for example null with local-only steps

SSH settings you will see often: ssh_username, ssh_timeout, ssh_private_key_file, ssh_agent_auth, ssh_interface (public or private IP), ssh_handshake_attempts. Cloud builders generate a throw-away key pair, so you usually only specify the user name.

What is an artifact?

An artifact is whatever a build leaves behind that you can use: an AMI ID per region, a Docker image ID and tags, a directory with a disk image, an Azure managed image name. Each builder defines its own artifact. A post-processor receives the artifact from the previous step and produces a new one (a tarball made from a disk image, say), and the original is deleted unless you say otherwise (keep_input_artifact = true).

Post-processor chains

A single post-processor block runs on the original artifact. To make steps depend on each other, use post-processors with an inner list: each entry feeds the next.

HCL
build {
  sources = ["source.docker.app"]

  # independent: each runs on the committed image
  post-processor "docker-tag" {
    repository = "registry.example.com/app"
    tags       = ["1.0"]
  }

  # a chain: tag first, then push the tagged image
  post-processors {
    post-processor "docker-tag" {
      repository = "registry.example.com/app"
      tags       = ["1.0"]
    }
    post-processor "docker-push" {}
  }
}

Reading rule: separate post-processor blocks run in parallel branches from the same artifact; the sequence inside post-processors { } is a pipeline. Pushing before tagging would push the wrong name, which is exactly why chains exist.

The manifest post-processor

Most pipelines need to know which image was built. Parsing log output is fragile; the built-in manifest post-processor writes a JSON file.

HCL
build {
  sources = ["source.amazon-ebs.ubuntu"]

  post-processor "manifest" {
    output     = "manifest.json"
    strip_path = true
    custom_data = {
      git_sha = var.git_sha
      env     = var.env
    }
  }
}

Its output has this shape (field values invented for illustration):

JSON
{
  "builds": [
    {
      "name": "ubuntu",
      "builder_type": "amazon-ebs",
      "build_time": 1700000000,
      "files": null,
      "artifact_id": "eu-west-1:ami-0123456789abcdef0",
      "packer_run_uuid": "00000000-0000-0000-0000-000000000000",
      "custom_data": { "git_sha": "abc1234", "env": "prod" }
    }
  ],
  "last_run_uuid": "00000000-0000-0000-0000-000000000000"
}

Notice the AMI artifact_id has the format region:ami-id and several regions are separated by commas. A later pipeline step can read it with jq:

Terminal
jq -r '.builds[-1].artifact_id | split(":") | .[1]' manifest.json

By default the manifest appends each run's build to the same file, so .builds[-1] is the newest; the last_run_uuid field tells you which run was last. Delete the file before a build if you want a clean one.

A stand-in for lifecycle and manifest

This Python stand-in runs the phases in order, supports a failure in any phase, always performs cleanup, and writes a manifest-like record on success. It models ordering and bookkeeping, not real cloud calls.

Python Stand-in: lifecycle with cleanup and manifest

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Observe that cleanup runs in both cases, and that nothing after the failing provisioner runs.

Error handling in practice

  • Default -on-error=cleanup removes everything. In a pipeline this is what you want.
  • Locally, -on-error=abort leaves the machine so you can SSH in and read logs; delete it afterwards.
  • If Packer itself is killed (Ctrl+C once is a graceful stop; a hard kill or power loss is not), build-time resources may be orphaned. Tag temporary resources via run_tags / tags on your cloud source and sweep old ones on a schedule.
  • -debug pauses between steps and is the right tool when you do not yet understand why a step fails.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Which phase happens just before post-processors run?

    After provisioning, the machine is stopped and the artifact is created; post-processors then work on it.
  2. 2. What is a Packer artifact?

    Artifacts are what builds produce. Post-processors receive them and can produce new ones.
  3. 3. How do you make docker-push run after docker-tag on the tagged image?

    The post-processors block forms a sequence; separate post-processor blocks run on the same original artifact.
  4. 4. Why use the manifest post-processor?

    The manifest gives scripts a reliable place to read the image ID instead of parsing logs.
  5. 5. Which communicator does a Windows image build normally use?

    WinRM is the classic Windows communicator; recent Windows images can also be provisioned over OpenSSH.

Summary

  • A build runs these phases per source: prepare, create machine, connect, provision, capture artifact, post-process, cleanup.
  • Communicators (SSH, WinRM, Docker) are how Packer reaches the machine.
  • An artifact is the output of a builder; post-processors transform artifacts.
  • post-processors { } builds a pipeline; separate post-processor blocks run independently.
  • The manifest post-processor writes a JSON record of artifact IDs, ideal for CI hand-off.
  • Failures trigger cleanup by default; use -on-error=abort and -debug to investigate.

Next: Multi-file templates and JSON