Skip to content
Tutorials Center
My learning

Topics

Lesson 6 of 37 ยท Architecture and Setup

Chef Architecture

Understand how Chef is built: workstation, Chef Infra Server, nodes, the pull model, organizations, and how cookbooks and node data move between them.

Chef has three kinds of machine: your workstation, the Chef Infra Server and the nodes. Understanding what each does, and which way data flows between them, makes every later topic easier, from bootstrapping to search. This lesson draws the picture and then models the central idea, the pull model, in a runnable Ruby stand-in. The real server and nodes cannot run on this site, so everything real is read-only.

The three parts

flowchart LR
  W["Workstation<br/>write and test code"] -- "knife / chef push" --> S["Chef Infra Server<br/>cookbooks, roles, node data"]
  N1["Node 1<br/>Chef Infra Client"] -- "pull cookbooks" --> S
  N2["Node 2<br/>Chef Infra Client"] -- "pull cookbooks" --> S
  N1 -- "report node data" --> S

Code flows from the workstation to the server; nodes pull it and report back.

  • Workstation. Your laptop. You write cookbooks, test them and upload them with knife or the Policyfile tools.
  • Chef Infra Server. A central hub that stores cookbooks, policies, roles, environments, data bags and a record (the node object) of every node. It also indexes this data so you can search it.
  • Nodes. The machines you manage. Each runs Chef Infra Client, which authenticates to the server, downloads what it needs, applies it and uploads the result.

A server is optional. With local mode and with Policyfile archives you can run Chef with no server at all, but the server is how most larger teams share data and search their fleet.

The pull model

In the pull model nodes ask for their configuration. The server never logs in to a node. That has real advantages:

  • You can add a node anywhere that can reach the server, even behind a firewall that allows only outbound traffic.
  • Nodes check in regularly (by default about every 30 minutes with some random delay), so drift is corrected automatically.
  • The server does no work per node while they converge, which helps scale.

Tools like Ansible mostly push instead: a control machine connects out to each host. Neither is better in all cases, but you should know which one you are using.

Organizations and authentication

A Chef Infra Server can host several organizations, each a separate namespace with its own cookbooks, nodes and users. Every request is signed with an RSA private key, and the server checks the signature, so nobody can impersonate a node. Users have *.pem keys on their workstation, nodes have a client key created when they are bootstrapped. The key file normally lives at /etc/chef/client.pem on the node.

Never commit .pem keys to Git. A leaked user key can change every node in the organization.

A real client.rb (read-only)

Each node has a small config file that says which server to talk to:

Ruby
log_level        :auto
chef_server_url  'https://chef.example.com/organizations/acme'
node_name        'web1.example.com'
validation_client_name 'acme-validator'

chef_server_url includes the organization name at the end. node_name is how the node appears in the server. You will set these up in the bootstrap lesson.

Stand-in: server and clients in Ruby

The stand-in below models a server that stores cookbooks and node records and clients that pull and report. It is plain Ruby, not the real Chef protocol.

Ruby Stand-in: the pull model

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with gets), it gets the next line from this box.

Notice who starts every conversation: the clients. The server only answers. Because the server keeps each node's data, you can later ask it "which nodes run nginx?", which is Chef search.

Search: the server as a database

When a node uploads its data, the server indexes it. Recipes and knife can search that index, for example to let a load balancer find every web server. The real syntax is shown read-only:

Ruby
web_servers = search(:node, 'role:web AND chef_environment:production')

The stand-in below mimics the idea on the data we collected, filtering nodes by a condition.

Ruby Stand-in: search the node index

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with gets), it gets the next line from this box.

Search turns your fleet into a queryable inventory, but it also means a recipe can behave differently depending on the data on the server. Remember that when you test with local mode.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. In Chef's pull model, who starts the conversation?

    Nodes pull cookbooks and report data. The server never logs in to a node.
  2. 2. Where do you normally write and test cookbooks?

    The workstation is your development machine, from which you upload finished work.
  3. 3. What is an organization on the Chef Infra Server?

    Organizations isolate data on a shared server, and the server URL ends with the organization name.
  4. 4. Why must .pem keys be kept out of Git?

    The keys authenticate requests. A leaked key is a leaked identity.

Summary

  • The workstation authors code, the server stores and indexes data, nodes pull and converge.
  • The pull model keeps the server out of each node and lets nodes check in regularly.
  • Requests are signed with RSA keys; organizations separate tenants.
  • The server indexes node data, so you can search the fleet.

Next: The Chef Client Run