Skip to content
Tutorials Center
My learning

Topics

Lesson 13 of 36 ยท Builders and Provisioners

Packer Shell and File Provisioners: Scripts, Uploads, Cleanup

Use Packer's shell, shell-local and file provisioners: inline commands, scripts, environment variables, execute_command, uploads and downloads, retries, ordering and image cleanup.

A provisioner is a step that changes the temporary machine. The two you will use most are shell, which runs commands inside the machine, and file, which copies files into it (or out of it). Together they cover most image recipes: upload a configuration file, run an install script, clean up. This lesson shows every option you need and the habits that keep scripts reliable.

Packer is not available on this site, so HCL and shell code is read-only; no real run output is shown. The Python stand-ins model script ordering and a cleanup checklist.

The shell provisioner

Three ways to give it commands:

HCL
# 1. inline: a list of commands joined and run in order
provisioner "shell" {
  inline = [
    "sudo apt-get update",
    "sudo apt-get install -y nginx",
  ]
}

# 2. script: one local file, uploaded and run
provisioner "shell" {
  script = "${path.root}/scripts/10-base.sh"
}

# 3. scripts: several files, run in the order listed
provisioner "shell" {
  scripts = [
    "${path.root}/scripts/10-base.sh",
    "${path.root}/scripts/20-app.sh",
  ]
}

Prefer script files for anything longer than a few lines: they can be linted with ShellCheck and run by hand. Use inline for one-liners.

Environment variables

HCL
provisioner "shell" {
  environment_vars = [
    "DEBIAN_FRONTEND=noninteractive",
    "APP_VERSION=${var.app_version}",
  ]
  scripts = ["${path.root}/scripts/20-app.sh"]
}

DEBIAN_FRONTEND=noninteractive stops apt from asking questions that nobody can answer. Use env in the script (set -u) to fail if a needed variable is missing.

Running with sudo

Cloud images log in as an unprivileged user. Either put sudo in every command, or override how scripts are launched:

HCL
provisioner "shell" {
  execute_command = "{{ .Vars }} sudo -E bash '{{ .Path }}'"
  environment_vars = ["DEBIAN_FRONTEND=noninteractive"]
  scripts = ["${path.root}/scripts/10-base.sh"]
}

execute_command is the template for launching the script: {{ .Path }} is the uploaded file and {{ .Vars }} expands environment variables. {{ .Vars }} puts the environment_vars in front of sudo, and -E keeps them for the script. Remember that an inline list is itself written to a temporary script and executed with this command, so execute_command applies to it too.

Write safe scripts

Terminal
#!/usr/bin/env bash
set -euo pipefail          # stop at the first error, undefined variable or failed pipe

export DEBIAN_FRONTEND=noninteractive
apt-get update
apt-get install -y --no-install-recommends nginx
systemctl enable nginx

Packer checks each script's exit status: non-zero fails the build. Without set -e, an early failure in the middle of a script may go unnoticed because only the last command's status counts.

Retries, delays and exit codes

HCL
provisioner "shell" {
  inline        = ["sudo apt-get update"]
  max_retries   = 3                  # try again if the command fails
  pause_before  = "10s"              # wait before the first try
  timeout       = "15m"              # give up if it takes too long
  valid_exit_codes = [0, 100]        # treat these exit codes as success
}

Cloud images often run cloud-init and unattended-upgrades at first boot, which hold the apt lock. A common pattern is to wait for cloud-init before touching packages:

HCL
provisioner "shell" {
  inline = ["cloud-init status --wait"]
}

Reboots

HCL
provisioner "shell" {
  inline            = ["sudo reboot"]
  expect_disconnect = true           # the connection will drop; that is expected
}

provisioner "shell" {
  pause_before = "30s"               # give the machine time to come back
  inline       = ["uname -r"]
}

expect_disconnect = true stops Packer from treating the dropped connection as a failure.

The file provisioner

HCL
provisioner "file" {
  source      = "${path.root}/files/nginx.conf"
  destination = "/tmp/nginx.conf"
}

provisioner "file" {
  source      = "${path.root}/files/site/"          # trailing slash: contents of the folder
  destination = "/tmp/site"
}

provisioner "file" {
  content     = templatefile("${path.root}/files/app.service.pkrtpl.hcl", { port = var.app_port })
  destination = "/tmp/app.service"
}

provisioner "file" {
  direction   = "download"
  source      = "/var/log/build-report.txt"
  destination = "output/build-report.txt"
}

Rules to remember:

  • The destination directory must exist and be writable by the SSH user. That is why files go to /tmp first and a following shell step uses sudo mv or sudo install to move them into /etc/....
  • A trailing slash on a directory source copies the contents; without it, the directory itself is copied into the destination.
  • content writes text from an expression instead of a local file. templatefile() renders a template with variables (Built-in functions).
  • direction = "download" copies from the machine to your computer, useful for reports and test results.

A typical pair:

HCL
provisioner "file" {
  source      = "${path.root}/files/app.service"
  destination = "/tmp/app.service"
}

provisioner "shell" {
  inline = [
    "sudo install -m 0644 /tmp/app.service /etc/systemd/system/app.service",
    "sudo systemctl enable app.service",
  ]
}

shell-local and other built-in provisioners

shell-local runs on your machine (or CI runner), not in the image: generate files, call APIs, run tests against the artifact path.

HCL
provisioner "shell-local" {
  inline = ["echo Building for ${source.name} > build-note.txt"]
}

Others: powershell and windows-shell/windows-restart for Windows images, breakpoint (pauses the build until you press Enter, good for exploring), and ansible, inspec, puppet-masterless, chef-solo from plugins (Ansible and other provisioners).

Order matters: a stand-in

Provisioners run in the order written, each completing before the next. This Python stand-in applies a list of provisioners to a fake machine, including the "upload to /tmp, then sudo move" pattern, and shows why a missing set -e can hide a failure.

Python Stand-in: provisioner order and exit codes

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Cleanup before the image is saved

Images should not carry build leftovers. A cleanup script, run as the last provisioner:

Terminal
#!/usr/bin/env bash
set -euo pipefail
apt-get clean
rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
truncate -s 0 /etc/machine-id            # regenerated on first boot
rm -f /etc/ssh/ssh_host_*                # regenerated on first boot (ensure sshd regenerates them)
rm -f /root/.bash_history ~/.bash_history
sync

Check your distribution's guidance before deleting host keys: some images regenerate them via cloud-init, others need a systemd unit. Never delete the SSH key Packer is currently using before the final step; do it last.

A stand-in cleanup checklist

Python Stand-in: audit an image for leftovers

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Why upload a config file to /tmp and then use a shell step with sudo?

    The file provisioner runs as the connection user, so privileged destinations need a follow-up sudo command.
  2. 2. What does expect_disconnect = true do?

    It prevents a reboot from being reported as a communicator failure.
  3. 3. Why start a bash script with set -euo pipefail?

    Without set -e only the last command's exit status counts, so earlier failures can be hidden.
  4. 4. Where does the shell-local provisioner run?

    shell-local executes on the workstation or CI runner.
  5. 5. What does a trailing slash on a directory source in the file provisioner mean?

    With the slash the contents are copied; without it the directory itself is placed inside the destination.

Summary

  • shell runs inline commands, one script or several scripts in order; file uploads or downloads files and directories.
  • Use environment_vars, execute_command (for sudo) and set -euo pipefail to make scripts predictable.
  • Upload to a writable location such as /tmp, then move into place with sudo in a shell step.
  • Use max_retries, pause_before, timeout and expect_disconnect for flaky or rebooting steps.
  • shell-local runs on your machine; finish with a cleanup script so images contain no build leftovers.

Next: Ansible and other provisioners