Skip to content
Tutorials Center
My learning

Topics

Lesson 30 of 38 ยท Testing and Code Management

Acceptance Testing with Litmus

Test Puppet code on real systems with Litmus: provision containers or VMs, install the agent, apply manifests, check idempotence and verify the result with serverspec-style checks.

Unit tests prove that a catalog contains the right resources. They cannot prove that the package really installs, that the config file is valid for the version of the software you ship, or that the service answers requests. For that you need an acceptance test: apply the code to a real operating system and observe what happens.

Litmus is the framework for this in the Puppet ecosystem. It provisions target machines (Docker containers, Vagrant boxes, cloud VMs), installs the agent, applies your manifest and lets you write assertions against the result. It is deliberately split into small steps so that you can repeat the slow ones rarely and the fast ones often. This lesson describes the workflow; the commands are read-only, and the runnable blocks are JavaScript stand-ins that model the "apply twice, expect no change" idempotence check at the heart of every acceptance suite.

The Litmus workflow

Litmus is driven by Rake tasks that the PDK template enables when you opt in. The typical development loop:

Terminal
pdk bundle exec rake 'litmus:provision_list[default]'   # create machines
pdk bundle exec rake litmus:install_agent                # install the Puppet agent
pdk bundle exec rake litmus:install_module               # build and install your module
pdk bundle exec rake litmus:acceptance:parallel          # run spec/acceptance tests
pdk bundle exec rake litmus:tear_down                    # remove machines

The provisioning targets come from provision.yaml:

YAML
default:
  provisioner: docker
  images: ['litmusimage/ubuntu:24.04', 'litmusimage/rockylinux:9']
vagrant:
  provisioner: vagrant
  images: ['generic/debian12']

Containers start in seconds and cover most package, file and service logic. Use a VM when the test needs a full init system, kernel settings, kernel modules or a firewall.

Between runs you normally keep the machines and only reinstall the module (litmus:reinstall_module), which makes the edit-test loop fast.

Writing an acceptance test

Acceptance tests live in spec/acceptance and use RSpec with helpers that run on the target:

Ruby
# spec/acceptance/webserver_spec.rb
require 'spec_helper_acceptance'

describe 'profile_base::webserver' do
  let(:pp) do
    <<~PUPPET
      class { 'profile_base::webserver':
        port => 8080,
      }
    PUPPET
  end

  it 'applies without errors' do
    apply_manifest(pp, catch_failures: true)
  end

  it 'is idempotent' do
    apply_manifest(pp, catch_changes: true)
  end

  describe package('apache2') do
    it { is_expected.to be_installed }
  end

  describe service('apache2') do
    it { is_expected.to be_running }
    it { is_expected.to be_enabled }
  end

  describe port(8080) do
    it { is_expected.to be_listening }
  end

  describe file('/etc/apache2/ports.conf') do
    its(:content) { is_expected.to match(%r{^Listen 8080$}) }
  end
end

The key helpers:

Helper Meaning
apply_manifest(pp, catch_failures: true) Apply; fail the test if any resource fails
apply_manifest(pp, catch_changes: true) Apply; fail if anything changes: this is the idempotence test
run_shell('cmd') Run a command on the target and return stdout, stderr, exit status
package, service, port, file Serverspec-style resource checks

The first apply makes changes; the second one must change nothing. A manifest that changes something every run (an exec without unless, a template with a timestamp) fails here and would otherwise create endless noise in reports.

Testing more than the happy path

Good suites add scenarios:

  • Change a parameter and re-apply: the service must pick up the new value and restart exactly when needed.
  • Remove a resource (ensure => absent) and verify cleanup.
  • Apply on an already half-configured machine, the situation of real servers.
  • Run once on every OS listed in provision.yaml.

Run slow suites in CI on a schedule or for release branches, and the quick unit tests on every commit.

flowchart LR
  P["provision targets"] --> I["install agent"]
  I --> M["install module"]
  M --> A1["apply (changes expected)"]
  A1 --> A2["apply again (no changes)"]
  A2 --> V["verify: package, service, port, file"]
  V --> T["tear down"]

Acceptance tests prove behavior on a real system, including idempotence.

Stand-ins: idempotence and verification

The stand-ins below model a tiny "machine" as an object and a resource applier that reports whether it changed anything. They are JavaScript stand-ins: they teach the check, not Litmus itself.

JS Stand-in: apply twice and compare

Press Run (or Ctrl + Enter) to see the console output here.

No output.

JS Stand-in: serverspec-style assertions

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Cost and speed

Acceptance tests are the slowest and most expensive layer, so keep them focused: one suite per module, a handful of scenarios, two or three target systems in CI. Use the unit tests for combinatorial coverage and Litmus for the questions only a real machine can answer. Together with lint and validate this forms a pyramid: many fast static checks, fewer unit tests, few acceptance tests.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does apply_manifest(pp, catch_changes: true) test?

    catch_changes fails when any resource changes, which on a second run signals non-idempotent code.
  2. 2. Which Litmus step installs your module on the targets?

    install_module builds the module and copies it to each target.
  3. 3. When is a VM target better than a container?

    Containers are fast but share the host kernel and often lack a full init system.
  4. 4. Which layer should have the most tests?

    A test pyramid keeps slow acceptance tests few and fast checks many.

Summary

  • Litmus provisions real or containerized machines, installs the agent and your module, and runs acceptance tests.
  • The core pattern is apply once with catch_failures, apply again with catch_changes, then verify the result.
  • Use serverspec-style helpers for packages, services, ports and files.
  • Containers for speed, VMs for kernel- and init-level behavior.
  • Keep acceptance suites small; leave breadth to unit tests.

Next: R10k and the Control Repo