Skip to content
Tutorials Center
My learning

Topics

Lesson 18 of 38 ยท Classes, Modules and Hiera

Hiera: Data Separated from Code

Keep data out of Puppet code with Hiera 5: hiera.yaml hierarchy, YAML data files, automatic parameter lookup, the lookup function, merge strategies and encrypted secrets.

Code says what to do; data says with which values. Mixing them makes code hard to reuse: a hard-coded NTP server, port or password ties the class to one site. Hiera is Puppet's built-in key-value lookup that keeps data in separate files and chooses the right value per node using a hierarchy. It is how the same class gets one setting in production and another in development, with no code change.

The idea

You ask for a key, say nginx::port. Hiera searches a list of data files from most specific to most general and returns the first match:

flowchart TB
    Q["lookup nginx::port"] --> L1["nodes/web1.example.test.yaml"]
    L1 -->|not found| L2["os/Debian.yaml"]
    L2 -->|not found| L3["environment: production.yaml"]
    L3 -->|not found| L4["common.yaml"]
    L4 -->|found 80| R["80"]

The most specific file wins; common.yaml supplies the fallback.

hiera.yaml

Hiera 5 is configured in hiera.yaml. One exists globally, one per environment (in the environment root, the one you usually edit) and optionally one per module:

YAML
# environments/production/hiera.yaml
---
version: 5
defaults:
  datadir: data
  data_hash: yaml_data
hierarchy:
  - name: "Per-node data"
    path: "nodes/%{trusted.certname}.yaml"
  - name: "Per-OS data"
    path: "os/%{facts.os.family}.yaml"
  - name: "Per-datacenter data"
    path: "datacenter/%{facts.datacenter}.yaml"
  - name: "Common data"
    path: "common.yaml"
  • version: 5 is required.
  • datadir is relative to the environment (the folder data/).
  • data_hash: yaml_data says the files are YAML (other backends: json_data, eyaml_lookup_key for encrypted values, plus custom ones).
  • Each hierarchy path may interpolate facts: %{facts.os.family}, %{trusted.certname}, %{::environment}. Levels whose variables are missing are skipped. Use trusted values for node-level paths so a node cannot choose its own data.

Data files

YAML
# data/common.yaml
---
nginx::port: 80
nginx::manage_service: true
ntp::servers:
  - 0.pool.ntp.org
  - 1.pool.ntp.org

# data/os/Debian.yaml
---
nginx::package: nginx-light

# data/nodes/web1.example.test.yaml
---
nginx::port: 8080

On web1, nginx::port is 8080 (node file) and nginx::package is nginx-light if it runs Debian, while a different node gets 80. A real boolean in YAML is true/false (no quotes). Strings that look like numbers or YAML special values should be quoted.

Using it

1. Automatic parameter lookup (the main way): include nginx makes Puppet look up nginx::port, nginx::package, and so on (Classes and Parameters). You write no lookup code at all.

2. The lookup function for values that are not class parameters:

PUPPET
$servers = lookup('ntp::servers')                       # fails if no value
$region  = lookup('site::region', String, 'first', 'eu') # type, strategy, default
$opts    = lookup({ name => 'app::options', default_value => {} })

3. The command line, for testing without a catalog:

Terminal
puppet lookup nginx::port --node web1.example.test
puppet lookup ntp::servers --explain

--explain prints which files were searched and where the value came from, the best tool for debugging "why is my value wrong?". For a real node, the --node option uses the facts stored on the server.

Merge strategies

By default the first match wins (first). For hashes and arrays you may want to combine values from several levels.

Strategy Behaviour
first Take the first value found (default)
unique Concatenate arrays from all levels and remove duplicates
hash Merge hashes from all levels; keys in more specific levels win; top-level keys only
deep Recursive merge of nested hashes and arrays

Set a strategy per key with lookup_options in the data, so it applies to automatic lookups too:

YAML
# data/common.yaml
lookup_options:
  profile::web::vhosts:
    merge: hash
  profile::base::packages:
    merge: unique

Now each data layer may add its own vhosts or packages, and the node sees the union. Without this, the node's own list would replace the common one entirely.

Secrets

Hiera data is committed to Git, so never store plain-text passwords. Options: encrypt values with hiera-eyaml (public-key encryption inside YAML, the server decrypts), use a secret store lookup backend (Vault and similar), or the Sensitive data type so values do not appear in logs and reports:

PUPPET
class app (Sensitive[String] $db_password) {
  file { '/etc/app/db.conf':
    content => Sensitive("password=${db_password.unwrap}\n"),
    mode    => '0600',
  }
}

Sensitive hides the value in output, but it still ends up in the compiled catalog on disk, so restrict access to the server.

Module and global layers

Lookups are resolved in three layers: the global layer (rare), then the environment layer, then each module layer. A module can ship its own defaults in its data/ and hiera.yaml: for example OS-specific package names. Environment data always beats module data, so site operators can override module defaults without editing the module.

Stand-ins: hierarchy and merge

Python Stand-in: first match wins

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Python Stand-in: unique and deep merge

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Notice that in the deep merge the shop port comes from the node layer while its root is kept from common.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What is the default Hiera merge behaviour?

    Unless lookup_options say otherwise, Hiera returns the first value found.
  2. 2. Which key does automatic lookup use for class nginx's port parameter?

    Keys are <class>::<parameter>.
  3. 3. Which command explains where a value came from?

    --explain prints the search path and the source of the value.
  4. 4. How should secrets be stored in Hiera data?

    Data in Git must not contain plain-text secrets.
  5. 5. Why prefer trusted.certname over a fact for per-node data?

    Self-reported facts could be manipulated; trusted data cannot.

Summary

  • Hiera separates data from code; hiera.yaml (version 5) defines a hierarchy of data files.
  • The first match wins by default; lookup_options selects unique, hash or deep merges.
  • Automatic parameter lookup (class::param) is the main way to use it; lookup() and puppet lookup --explain help elsewhere.
  • Protect secrets with eyaml, a secret backend and Sensitive.
  • Environment data overrides module data, which overrides nothing global.

Next: Facter and Core Facts