Skip to content
Tutorials Center
My learning

Topics

Lesson 23 of 36 ยท Post-processors and Pipelines

Post-processors: What They Do and How to Chain Them

Understand Packer post-processors: what an artifact is, the post-processor and post-processors blocks, sequences versus parallel steps, only/except and keep_input_artifact.

A provisioner changes the machine while it is running. A post-processor works after the machine has been turned into an artifact (an AMI, a disk file, a container image). Post-processors compress, convert, tag, upload, record or otherwise take that artifact one step further. They are optional, but nearly every real pipeline uses at least one, often the manifest post-processor that records what was built.

Packer cannot run on this site. HCL examples are read-only; the runnable examples are JavaScript stand-ins that model how artifacts flow through post-processors.

Artifacts

When a builder finishes it hands Packer an artifact. Each builder defines its own kind: an AMI id per region, a path to a qcow2 file, a Docker image id. The artifact has an id and a string describing it. A post-processor receives the artifact, does its work and usually returns a new artifact (or the same one). By default the input artifact is deleted once a later post-processor in the chain consumed it, for example the .vmdk is removed after the vagrant box is created, unless you set keep_input_artifact = true.

Two block forms

A single post-processor sits directly in the build block:

HCL
build {
  sources = ["source.qemu.debian"]

  post-processor "checksum" {
    checksum_types = ["sha256"]
    output         = "output/{{.BuildName}}.{{.ChecksumType}}"
  }
}

Several post-processors that should run in sequence, each receiving the previous one's artifact, go inside a post-processors block (plural):

HCL
build {
  sources = ["source.docker.app"]

  post-processors {
    post-processor "docker-tag" {
      repository = "registry.example.com/app"
      tags       = ["1.0.0"]
    }
    post-processor "docker-push" {}
  }
}

If you wrote docker-tag and docker-push as two separate post-processor blocks, they would run in parallel, each starting from the original artifact, and the push would not find the tagged image. The plural block is the sequence.

flowchart TD
  A["Artifact"] --> S1["post-processor block 1"]
  A --> S2["post-processor block 2"]
  S1 --> R1["artifact 1"]
  A --> SEQ["post-processors block"]
  SEQ --> X["step a"] --> Y["step b"] --> R2["artifact 2"]

Separate blocks fan out from the original artifact. Steps inside one plural block form a line.

Common post-processors

Name Purpose
manifest Writes a JSON file listing every artifact (see manifest and metadata)
checksum Computes hashes of file artifacts
compress Packs files into tar.gz, zip, etc.
shell-local Runs a command on the machine running Packer
artifice Replaces the artifact with files or ids you choose
vagrant Wraps a VM into a .box
docker-tag, docker-push, docker-save, docker-import Container image handling
amazon-import Imports a local disk as an AMI
googlecompute-export, googlecompute-import Move images to and from GCS

Controlling when they run

  • only / except. only = ["amazon-ebs.base"] limits a post-processor to certain sources; except excludes them. Names use the type.name form.
  • keep_input_artifact. Prevents deletion of the artifact the step consumed. Without it a compress step removes the uncompressed file.
  • name. Gives the step a label so logs and -only flags can refer to it.
  • Failure behavior. If a post-processor fails, the build fails. Artifacts already created remain in the cloud, so clean them up or make the step idempotent.

Parallel builds, one chain

If a build has several sources (for example an AMI and a GCE image), each source gets its own artifact and its own run of the post-processor chain. A manifest step therefore appends one entry per source. That is exactly why manifest is the standard way to hand results to later pipeline steps.

Try it: simulate a chain

This stand-in passes an artifact through a list of steps and prints what survives, to show the "input artifact is consumed" rule.

JS Post-processor chain simulator (stand-in)

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Try it: sequence versus parallel

JS Why docker-push needs the plural block (stand-in)

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. How do you make two post-processors run one after the other, each using the previous artifact?

    The plural block is a sequence; separate singular blocks run in parallel from the original artifact.
  2. 2. What happens to the input artifact of a compress step by default?

    Consumed artifacts are removed by default to save space.
  3. 3. Which post-processor records all produced artifacts in a JSON file?

    manifest writes packer-manifest.json by default with an entry per build.
  4. 4. What does only = ["amazon-ebs.base"] on a post-processor do?

    only restricts the step to the listed sources; except does the opposite.

Summary

  • Post-processors act on artifacts after the build; provisioners act on the running machine.
  • A post-processors block chains steps; separate blocks run in parallel.
  • Consumed input artifacts are deleted unless keep_input_artifact = true.
  • only and except select sources; manifest is the standard record of results.

Next: The manifest post-processor and image metadata