Skip to content
Tutorials Center
My learning

Topics

Lesson 32 of 37 ยท Scale and Operations

The chef-client Run, Handlers and Reporting

Understand each phase of a chef-client run, how the two-phase compile and converge model works, notifications, report handlers and run status, with a runnable Ruby model.

When chef-client runs, a lot happens in a fixed order. Knowing that order explains many puzzling behaviors: why a Ruby statement in a recipe runs before a package installs, why a notification arrives at the end, why a failed resource stops the rest. It also tells you where to attach handlers: small pieces of code that run when a run finishes, fails or changes something, so you can report to Slack, a ticket system or a metrics store.

This lesson walks through the run phases, the compile versus converge distinction, notifications and their timing, and handlers. The runnable examples model the two phases in plain Ruby, because the real client cannot run here.

Phases of a run

flowchart TD
  A["Start: read client.rb, load node data (Ohai)"] --> B["Authenticate with the server, sync cookbooks"]
  B --> C["Expand the run list, build attributes"]
  C --> D["Compile: run recipe code, build the resource collection"]
  D --> E["Converge: execute each resource in order"]
  E --> F["Delayed notifications"]
  F --> G["Report to the server, run handlers"]

Steps D and E are the heart of the run. Everything else prepares or reports.

  1. Ohai collects facts (OS, CPU, memory, network) into automatic attributes.
  2. The client authenticates, saves a first copy of the node, and downloads the cookbooks the run list needs.
  3. Attributes are merged and the run list is expanded.
  4. Compile phase: recipes are evaluated as Ruby from top to bottom. Each resource block is added to the resource collection, not executed.
  5. Converge phase: the client walks the collection and, for each resource, checks the current state and acts if it differs from the desired state.
  6. Notifications that were delayed run after converge.
  7. The node is saved and run data is reported; handlers fire.

Why two phases matter

Plain Ruby in a recipe runs at compile time, resources run at converge time. This causes a classic puzzle:

Ruby
package 'nginx'

# Runs during compile, BEFORE the package is installed:
if ::File.exist?('/usr/sbin/nginx')
  log 'nginx found'
end

On a fresh machine the file does not exist when the if is evaluated, so the log resource is never added. Fixes: use a guard on the resource (only_if { ::File.exist?('/usr/sbin/nginx') }, evaluated at converge time), or wrap Ruby in ruby_block, or force an earlier action with run_action.

Ruby Stand-in: compile versus converge

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with gets), it gets the next line from this box.

The plain Ruby line during compile reports false, while the ruby_block sees true, because it runs after the package resource. A second run reports "already installed", which is idempotence again.

Notifications and subscriptions

A resource can ask another to act when it changes. By default notifications are delayed: they are queued and run once, at the end of the converge phase.

Ruby
template '/etc/nginx/nginx.conf' do
  source 'nginx.conf.erb'
  notifies :reload, 'service[nginx]', :delayed      # at the end, once
end

execute 'initialize database' do
  command '/opt/myapp/bin/init-db'
  notifies :start, 'service[myapp]', :immediately   # right after this resource
end

service 'nginx' do
  subscribes :restart, 'package[nginx]'             # same idea, written from the other side
  action [:enable, :start]
end

:before runs the target before the resource acts. Because delayed notifications are de-duplicated, three template changes still cause one restart.

Failure behavior

If a resource raises an error, the converge stops at that point, later resources do not run, and the client exits non-zero. Options:

  • ignore_failure true on a resource lets the run continue (use rarely).
  • retries 3 and retry_delay 5 retry flaky actions such as downloads.
  • Delayed notifications still run only if the converge reaches the end; on a failure they are skipped.

Handlers

Handlers are classes that run at the end of a run. There are three kinds:

  • Start handlers: at the very beginning, before anything else.
  • Report handlers: after a successful run.
  • Exception handlers: after a failed run.

A report handler gets run_status with the list of updated resources, the elapsed time and the node:

Ruby
# libraries/slack_handler.rb
require 'chef/handler'

class MyCompany::ChangeReporter < Chef::Handler
  def report
    return if run_status.updated_resources.empty?

    names = run_status.updated_resources.map(&:to_s).join(', ')
    message = "#{node.name}: #{run_status.updated_resources.size} resources changed (#{names}) in #{run_status.elapsed_time.round(1)}s"
    Chef::Log.info(message)
    # post message to chat or a metrics API here
  end
end

You enable it from client.rb (report_handlers << MyCompany::ChangeReporter.new) or with the chef_handler resource inside a cookbook, so that the handler is installed by Chef itself. Modern setups often use the built-in Chef Automate data collector (data_collector.server_url) to send every run's report to a dashboard without custom handler code.

Ruby Stand-in: a report handler and an exception handler

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with gets), it gets the next line from this box.

Practical tips

  • Read the run output from the top: the resource list shows what changed, with diffs.
  • chef-client -l debug shows attribute and search details; --why-run shows what would change.
  • In production, run Chef as a service or on a timer (interval, splay; see the next lesson), not by hand.
  • Use Chef::Log for messages instead of puts; the log level controls visibility.
  • Keep compile-phase Ruby small, and put real work into resources.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. During which phase do resource blocks perform their actions?

    At compile time resources are only added to the collection. They act during converge.
  2. 2. When do :delayed notifications run?

    Delayed notifications are queued and executed at the end of the converge phase.
  3. 3. A report handler runs after what?

    Report handlers run after successful runs; exception handlers run after failures.
  4. 4. Why does a plain Ruby if File.exist? check before a package resource often give false on first run?

    Recipe Ruby executes during compile. Use guards or ruby_block so the check happens during converge.

Summary

  • A run: Ohai, authenticate and sync cookbooks, merge attributes, compile, converge, delayed notifications, save and report.
  • Compile builds the resource collection; converge executes it, which explains many ordering surprises.
  • Notifications are immediate, delayed (default) or before; failures stop the converge unless you retry or ignore.
  • Handlers report results (success, failure, start); Chef Automate's data collector covers most reporting needs.

Next: Splay, Intervals and Scaling Chef Runs