Skip to content
Tutorials Center
My learning

Topics

Lesson 18 of 36 ยท Variables and HCL

Packer Data Sources and Handling Secrets Safely

Look up the newest AMI with a Packer data source, read secrets from AWS Secrets Manager or Vault, mark values sensitive, and keep credentials out of logs and images.

Some values cannot be written into a template because they change over time or must not be stored in Git: the ID of the newest Ubuntu image, an API token, a database password. Data sources fetch such values when the build starts, and sensitive markings keep them out of Packer's output. This lesson shows both, and then the part that matters most: how to stop secrets from leaking into logs, shell history and the finished image.

Packer and cloud APIs are not available on this site, so HCL is read-only and no real lookup results are shown; AMI IDs and secrets in examples are placeholders. The Python stand-ins model redaction and a leak scan.

Data sources

A data block asks a plugin to read something and exposes the result as attributes:

HCL
data "amazon-ami" "ubuntu" {
  region = var.region

  filters = {
    name                = "ubuntu/images/hvm-ssd-gp3/ubuntu-noble-24.04-amd64-server-*"
    root-device-type    = "ebs"
    virtualization-type = "hvm"
  }

  most_recent = true
  owners      = ["099720109477"]
}

source "amazon-ebs" "web" {
  region        = var.region
  source_ami    = data.amazon-ami.ubuntu.id
  instance_type = "t3.micro"
  ssh_username  = "ubuntu"
  ami_name      = "web-${local.timestamp}"
  tags = {
    BaseAMI     = data.amazon-ami.ubuntu.id
    BaseAMIName = data.amazon-ami.ubuntu.name
  }
}

Reference form: data.<type>.<name>.<attribute>. amazon-ami exposes id, name, creation_date, owner, owner_name and tags. Compared with source_ami_filter inside the source, a data source has two advantages: you can reuse the result in several places (the AMI ID in tags, in a manifest, in a second source) and you can see it in packer console.

When are data sources evaluated? At the start of the run, before any machine is created. That means a failed lookup stops the build early and cheaply, and values are identical for every source in the run. A data source cannot depend on a running machine.

Other data sources come from plugins, for example amazon-secretsmanager, amazon-parameterstore and amazon-ami from the amazon plugin, as well as sources for HashiCorp Vault, Git or HTTP endpoints from other plugins. Declare them in required_plugins like any plugin.

Reading a secret

HCL
data "amazon-secretsmanager" "registry" {
  name = "prod/image-build/registry"
  key  = "token"
}

locals {
  registry_token = data.amazon-secretsmanager.registry.value
}

AWS credentials used by Packer decide whether the lookup works, so grant the CI role secretsmanager:GetSecretValue for that one secret and nothing wider. Alternatives are the vault(...) function, environment variables supplied by your CI system (PKR_VAR_registry_token) and aws_secretsmanager(...) for one-off reads. Pick one mechanism per project and document it in the README.

Sensitive values

HCL
variable "registry_token" {
  type      = string
  sensitive = true
}

local "auth_header" {
  expression = "Authorization: Bearer ${var.registry_token}"
  sensitive  = true
}

A sensitive variable or local is replaced with <sensitive> when Packer prints it, and values derived from it are treated as sensitive too. That protects casual screen sharing and normal CI logs. What it does not do:

  • It does not encrypt anything.
  • It cannot hide a secret that a script prints, such as echo $TOKEN or set -x output; the remote command's output is streamed back as plain text. Packer does mask known sensitive values in output lines it prints, but you should never rely on that as your only protection.
  • It does not remove the value from an image if you wrote it there.
  • Debug logs (PACKER_LOG=1) can include more than normal output. Treat them as confidential.

How masking works (a stand-in)

This Python stand-in hides known secret values in log lines, like a UI would. The last line shows the weakness: an encoded version of the secret is not recognised.

Python Stand-in: redacting secrets in output

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The Base64 form slips through, which is why you design so that secrets never reach logs in the first place.

Rules for secrets in image builds

  1. Do not bake secrets into images. Anything in the image (files, environment, shell history, AMI user data) is visible to everyone who can launch or copy the image. Images are shared, copied across regions and kept for years.
  2. Use secrets only at build time, and remove them before the image is saved: delete credential files, unset variables, clear history, and remove ~/.docker/config.json, .netrc, .npmrc and similar files.
  3. Prefer short-lived credentials: OIDC federation, role assumption, tokens with a one-hour lifetime. A leaked 10-minute token is a smaller problem than a leaked permanent key.
  4. Fetch at runtime for runtime needs. An application's database password belongs in a secret manager that the running server reads using its instance role, not in the image.
  5. Pass secrets through environment variables of a single provisioner, not on a command line that appears in process listings and logs:
HCL
provisioner "shell" {
  environment_vars = ["REGISTRY_TOKEN=${var.registry_token}"]
  script           = "${path.root}/scripts/pull-private-package.sh"
}
  1. Disable tracing in scripts that use secrets (set +x), and avoid curl -v.
  2. Scan finished images and the template repository for secrets in your pipeline (Testing and security chapter).
  3. Never commit *.auto.pkrvars.hcl or .pkrvars.hcl files containing real values; add them to .gitignore.

Cleaning up after a secret-using step

Terminal
#!/usr/bin/env bash
set -euo pipefail
rm -f ~/.docker/config.json ~/.npmrc ~/.netrc
unset REGISTRY_TOKEN
history -c || true

Run it as a provisioner right after the step that needs the secret, not only at the very end: a failing later step could otherwise leave the file in a machine you chose to keep with -on-error=abort.

A leak scan stand-in

Python Stand-in: scan files for secret-looking content

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Real scanners (gitleaks, trufflehog, cloud-native tools) use many more patterns, but the idea is the same.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. When are data sources evaluated?

    Data sources are read up front, so lookup failures stop the build before anything costs money.
  2. 2. What does sensitive = true guarantee?

    It is a display feature only; the value can still leak through scripts, logs or files.
  3. 3. Where should a running application's database password live?

    Images are widely copied and long-lived, so runtime secrets belong in a secret manager.
  4. 4. How do you reference the ID returned by data "amazon-ami" "ubuntu"?

    Data source references have the form data.<type>.<name>.<attribute>.
  5. 5. Which is the safest way to give a script a token during a build?

    Environment variables avoid process-list exposure, and cleanup removes leftover credential files.

Summary

  • data blocks fetch values at the start of a run and are referenced as data.<type>.<name>.<attribute>; amazon-ami finds the newest base image.
  • Secrets can come from Secrets Manager, Vault or CI-provided PKR_VAR_* variables; give the build role the minimum permissions.
  • sensitive hides values in Packer's output only; scripts, debug logs and files can still leak them.
  • Never bake secrets into images: use short-lived credentials at build time, clean up right after use and scan finished images.
  • Keep value files with real secrets out of Git.

Next: Building AMIs on AWS with amazon-ebs