Skip to content
Tutorials Center
My learning

Topics

Lesson 31 of 37 ยท Scale and Operations

Chef Infra Server, Knife and Search

Learn what the Chef Infra Server stores, the essential knife commands for nodes, cookbooks, roles and data bags, and how search queries find nodes, with a Ruby model of search.

The Chef Infra Server is the hub of a classic Chef setup. It stores cookbooks, roles, environments, data bags and a record of every node (its attributes and the last run). Nodes pull from it; you manage it with the command-line tool knife. One of the server's best features is search: recipes and engineers can ask "which nodes run the web role in production?" and get an answer from live data.

This lesson shows what lives on the server, the knife commands you use every day, and how to write search queries. You will also run a model of the search language to practice query logic. The real server and knife are not available on this site, so their commands are read-only.

What the server stores

flowchart TD
  U["Workstation: knife, chef CLI"] -->|"upload"| S["Chef Infra Server"]
  N1["node web01"] -->|"chef-client pulls cookbooks, saves node data"| S
  N2["node db01"] -->|"chef-client"| S
  S --> I["Search index"]
  S --> D["Cookbooks, roles, environments, data bags, policies"]

Nodes pull their configuration and push their attributes back; the server indexes everything for search.

Authentication is by RSA key pairs: each client (node or user) signs its requests with a private key, and the server checks the signature with the stored public key. An organization is a separate tenant with its own cookbooks and nodes, and access is controlled with permissions (ACLs) and groups.

Everyday knife commands

Configuration goes in .chef/config.rb (knife.rb) in your repository: server URL, your user name and key. Then:

Terminal
# nodes
knife node list
knife node show web01
knife node show web01 -a ipaddress -a platform
knife node run_list set web01 'role[web]'
knife node delete web01 -y

# bootstrap a new machine and run Chef on it for the first time
knife bootstrap web02.example.com -U ubuntu --sudo -N web02 -E production -r 'role[web]'

# cookbooks and policies
knife cookbook upload myapp
knife cookbook list
knife cookbook delete myapp 1.0.0
knife cookbook show myapp

# roles, environments and data bags
knife role from file roles/web.rb
knife environment from file environments/production.rb
knife data bag from file users data_bags/users/alice.json

# remote shell on many nodes
knife ssh 'role:web AND chef_environment:production' 'sudo chef-client' -x ubuntu

knife ssh is the bridge between search and action: it runs a search, then connects to every matching node. Treat it with care; a typo in the query targets the wrong machines.

Search basics

Search uses a Lucene-like language over the node data (attributes, run list, roles, environment). The pieces:

Query Meaning
role:web Nodes with role[web] in their expanded run list
chef_environment:production Nodes in an environment
platform:ubuntu Match an attribute (the automatic one)
name:web* Wildcard: * any characters, ? one character
role:web AND chef_environment:production Both must match
role:web OR role:cache Either
NOT role:db Negation
role:web AND NOT chef_environment:dev Combination
ipaddress:10.0.* Wildcard inside a value
network_interfaces_eth0_addresses:* Any value exists

On the command line: knife search node 'role:web AND platform:ubuntu' -a ipaddress. In a recipe:

Ruby
web_nodes = search(:node, 'role:web AND chef_environment:' + node.chef_environment)

template '/etc/haproxy/haproxy.cfg' do
  source 'haproxy.cfg.erb'
  variables(backends: web_nodes.map { |n| { name: n.name, ip: n['ipaddress'] } })
end

Search can also look in data bags: search(:users, 'groups:ops') as you saw earlier, and in roles, environments and clients.

The model below parses a small subset of the language (AND, OR, NOT, field:value with * and ? wildcards) and applies it to a list of node hashes. Its precedence is simplified: OR binds loosest, then AND, then NOT. It is meant to build intuition, not to replace the real server.

Ruby Stand-in: evaluate search queries over nodes

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with gets), it gets the next line from this box.

Study the results: NOT role:db removes the database node from the production set, and name:web* matches both web nodes regardless of environment. These are the same decisions a real knife ssh query makes before it opens connections.

Search in recipes: patterns and traps

  1. Self-registering load balancers. The load balancer recipe searches for web nodes and builds its backend list. When a new web node converges, the next load balancer run includes it.
  2. Wait for the index. A node's data becomes searchable shortly after its run, not instantly. A node that has not finished its first run may not appear yet, so recipes should handle an empty list.
  3. Search only what you need. Use the filter_result option to fetch a few attributes instead of full node objects:
Ruby
search(:node, 'role:web', filter_result: { 'name' => ['name'], 'ip' => ['ipaddress'] })
  1. Restrict to the environment. Forgetting chef_environment: makes staging machines join the production load balancer.
  2. Local mode has limits. With chef-client -z, search works only over the data in the local repository.

Good knife hygiene

  • Keep a config per organization and use knife ... --config or CHEF_ environment settings rather than editing one file back and forth.
  • Prefer knife ... -a attribute over dumping whole nodes.
  • Do not run knife ssh against *:* unless you mean every machine.
  • For large fleets, use Chef Automate or the Chef API from scripts rather than shelling out many knife commands.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does the query role:web AND chef_environment:production return?

    AND requires both conditions to match.
  2. 2. What does knife ssh do?

    knife ssh runs a search query and connects to every matching node to run the command.
  3. 3. Why should a load balancer search include chef_environment?

    Without it, nodes from other environments that match the role would be included.
  4. 4. Which wildcard matches exactly one character in a search term?

    • matches any number of characters and ? matches one.

Summary

  • The Chef Infra Server stores cookbooks, roles, environments, data bags, policies and node data, and indexes them for search.
  • knife manages nodes, cookbooks, roles, environments and data bags; knife bootstrap enrolls a machine and knife ssh runs commands on search results.
  • Search queries combine field:value terms with AND, OR, NOT and wildcards, in recipes or on the command line.
  • Restrict searches by environment, handle empty results, and use filter_result for performance.

Next: The chef-client Run, Handlers and Reporting