Skip to content
Tutorials Center
My learning

Topics

Lesson 24 of 29 ยท Projects

Project: Multi-Tier Stack with Roles and Vault

Build a load balancer, app servers and a database with Ansible roles, separate dev and prod inventories, group_vars, Vault secrets, play ordering and delegate_to.

Real systems have tiers: a load balancer receives traffic, application servers do the work, and a database keeps the data. Each tier is configured differently, but they depend on one another: the balancer must know the application addresses, and the application must know the database address and password. This project ties together almost everything in the tutorial: inventories per environment, group_vars, roles, Vault for secrets, play ordering and delegate_to.

You will build the structure for a three-tier stack that can be deployed to dev (small, cheap) and prod (bigger) from the same code. If any piece feels unfamiliar, revisit Inventory, Roles and Project Layout, Ansible Vault and Secrets and Handlers, Tags and Task Control.

Ansible, Vault, HAProxy and PostgreSQL cannot run on this site, so the project files are read-only. The runnable examples are JavaScript stand-ins: one parses an inventory and generates the load balancer configuration from it, the other shows how a Vault-backed variable resolves in each environment. They model the idea, not the real tools.

The architecture

flowchart LR
  U["Users"] --> LB["lb1<br/>HAProxy"]
  LB --> A1["app1"]
  LB --> A2["app2"]
  A1 --> DB[("db1<br/>PostgreSQL")]
  A2 --> DB

Traffic enters through one balancer, spreads over the app servers, and the apps share one database.

Project layout

One repository holds everything. Environments differ only in their inventory directory; the roles and the playbook are shared.

TEXT
stack/
  ansible.cfg
  site.yml
  inventories/
    dev/
      hosts.ini
      group_vars/
        all/
          vars.yml
          vault.yml        <- encrypted
        app.yml
    prod/
      hosts.ini
      group_vars/
        all/
          vars.yml
          vault.yml        <- encrypted
        app.yml
  roles/
    loadbalancer/  (tasks, templates, handlers, defaults)
    app/
    database/

Selecting an environment is just choosing the inventory: -i inventories/dev or -i inventories/prod. Because the inventory is a directory, Ansible reads its hosts.ini and the group_vars beside it automatically (see Inventory: Hosts and Groups).

Inventories and group variables

INI
# inventories/prod/hosts.ini
[lb]
lb1 ansible_host=10.0.0.10

[app]
app1 ansible_host=10.0.1.11
app2 ansible_host=10.0.1.12
app3 ansible_host=10.0.1.13

[db]
db1 ansible_host=10.0.2.5

The dev inventory has the same group names with one machine each. Because groups are named the same everywhere, the playbook and roles never need to know which environment they are in.

YAML
# inventories/prod/group_vars/all/vars.yml
app_port: 8080
db_name: shop
db_user: shop
db_password: "{{ vault_db_password }}"
YAML
# inventories/prod/group_vars/app.yml
app_workers: 8

dev would have app_workers: 2. Anything that differs between environments belongs in these files; anything shared belongs in the roles' defaults.

Secrets with Vault

Secrets must not sit in plain text in Git. Ansible Vault encrypts a file or a value. A well-known convention keeps the plain-text variable file readable but points at an encrypted one:

  • vars.yml (readable) says db_password: "{{ vault_db_password }}".
  • vault.yml (encrypted) defines only vault_db_password.

Everyone can see which variables exist and grep for where db_password is used, while only people with the Vault password can read the value. Create the encrypted file and run the stack like this:

Terminal
ansible-vault create inventories/prod/group_vars/all/vault.yml
ansible-playbook -i inventories/prod site.yml --ask-vault-pass

For an environment with different people or stricter rules, use separate Vault IDs so the dev password cannot open prod secrets, for example --vault-id prod@prompt. In pipelines, pass a password file or a secret provided by the CI system (see Ansible Vault and Secrets). Tasks that handle the value should carry no_log: true, so it never appears in output.

Stand-in: how the secret resolves per environment

The model below holds a readable variable file and a pretend vault for each environment. Resolving db_password needs the vault; without it the lookup fails, which is the point of the separation.

JS Vault indirection per environment

Press Run (or Ctrl + Enter) to see the console output here.

No output.

The roles

Each role does one tier's job and takes its settings from variables.

database

YAML
# roles/database/tasks/main.yml
- name: Install PostgreSQL
  ansible.builtin.apt:
    name:
      - postgresql
      - python3-psycopg2
    state: present
    update_cache: true
    cache_valid_time: 3600

- name: Listen on the private address
  community.postgresql.postgresql_set:
    name: listen_addresses
    value: "localhost,{{ ansible_host }}"
  become_user: postgres
  notify: restart postgresql

- name: Allow the app servers to connect
  community.postgresql.postgresql_pg_hba:
    dest: "/etc/postgresql/{{ postgresql_version }}/main/pg_hba.conf"
    contype: host
    databases: "{{ db_name }}"
    users: "{{ db_user }}"
    source: "{{ hostvars[item].ansible_host }}/32"
    method: scram-sha-256
  loop: "{{ groups['app'] }}"
  notify: reload postgresql

- name: Create the application database
  community.postgresql.postgresql_db:
    name: "{{ db_name }}"
  become_user: postgres

Each role also has a handlers/main.yml that defines the handlers it notifies (restart postgresql, reload postgresql, restart shop, reload haproxy); they are omitted here for space.

The path and package names depend on your distribution and PostgreSQL version, so postgresql_version is a role default (for example 16) that you adjust. The loop over groups['app'] is the key idea: the database role reads the inventory to learn who may connect, so adding an app server to the inventory opens the right firewall rule on the next run with no edits to the role.

app

YAML
# roles/app/tasks/main.yml
- name: Create the application database role
  community.postgresql.postgresql_user:
    name: "{{ db_user }}"
    password: "{{ db_password }}"
    db: "{{ db_name }}"
    priv: ALL
  become_user: postgres
  delegate_to: "{{ groups['db'][0] }}"
  run_once: true
  no_log: true

- name: Render the application environment file
  ansible.builtin.template:
    src: app.env.j2
    dest: /opt/shop/shared/config.env
    owner: shop
    group: shop
    mode: "0640"
  no_log: true
  notify: restart shop

delegate_to runs a task on another machine while keeping the current host's variables. Here the app role needs to create the database user, which must happen on the database server, so the task is delegated to the first host in the db group. run_once: true runs it a single time for the whole play instead of once per app server. Deploying the application itself, with release directories and a rolling update, is the subject of Project: Deploy an Application with Rolling Updates.

The environment file template builds the connection string from inventory facts:

JINJA
# roles/app/templates/app.env.j2
APP_PORT={{ app_port }}
APP_WORKERS={{ app_workers }}
DATABASE_URL=postgresql://{{ db_user }}:{{ db_password }}@{{ hostvars[groups['db'][0]].ansible_host }}:5432/{{ db_name }}

loadbalancer

JINJA
# roles/loadbalancer/templates/haproxy.cfg.j2
frontend http_in
    bind *:80
    default_backend app

backend app
    balance roundrobin
    option httpchk GET /health
{% for host in groups['app'] %}
    server {{ host }} {{ hostvars[host]['ansible_host'] }}:{{ app_port }} check
{% endfor %}
YAML
# roles/loadbalancer/tasks/main.yml
- name: Install HAProxy
  ansible.builtin.apt:
    name: haproxy
    state: present
    update_cache: true
    cache_valid_time: 3600

- name: Render the balancer configuration
  ansible.builtin.template:
    src: haproxy.cfg.j2
    dest: /etc/haproxy/haproxy.cfg
    owner: root
    group: root
    mode: "0644"
    validate: "haproxy -c -f %s"
  notify: reload haproxy

- name: Make sure HAProxy runs
  ansible.builtin.service:
    name: haproxy
    state: started
    enabled: true

The template never mentions an address. It walks the app group, so the same file produces a one-server backend in dev and a three-server backend in prod.

Stand-in: generate the balancer configuration from inventory

This program parses a small INI inventory (the same shape as above) for both environments, then renders the backend block, imitating the template's for loop over groups['app'].

JS Balancer config from the inventory

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Add an app4 line to the production inventory text and run again: the balancer config gains a line. That is the entire scaling story for this tier.

The site playbook and play order

YAML
# site.yml
- name: Database tier
  hosts: db
  become: true
  roles:
    - database

- name: Application tier
  hosts: app
  become: true
  serial: 1
  roles:
    - app

- name: Load balancer tier
  hosts: lb
  become: true
  roles:
    - loadbalancer

Order matters, and plays are how you express it. A play runs completely, on all its hosts, before the next play starts. The database must exist before the apps start and create a user in it; the apps should be healthy before the balancer sends them traffic. Reading the file top to bottom gives the dependency order: data, then logic, then entry point. (When you tear a stack down or update only the balancer, you can run one play with --limit lb, or use --tags as in Handlers, Tags and Task Control.)

serial: 1 in the application play updates one server at a time, so the balancer's health check always has other healthy servers to use. For a safer rollout, add max_fail_percentage: 0 as in Ansible with Docker, Terraform and CI/CD.

Running the project

Terminal
ansible-inventory -i inventories/dev --graph
ansible-playbook -i inventories/dev site.yml --check --diff --ask-vault-pass
ansible-playbook -i inventories/dev site.yml --ask-vault-pass
ansible-playbook -i inventories/prod site.yml --ask-vault-pass --limit app

ansible-inventory --graph confirms that Ansible sees the groups you expect. Practise on dev first, as the environments differ only in data, then promote the same unchanged code to prod. A second run of the whole playbook should end with changed=0 on every host.

Extend the project

  • Add the baseline from Project: Security Baseline Role as a play that runs on all before everything else.
  • Add a second balancer and a keepalived role for failover.
  • Replace the static inventory with a dynamic inventory that groups cloud instances by tag.
  • Add a site-smoke.yml playbook that uses uri against the balancer's address and fails the pipeline if the stack does not answer.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why does vars.yml say db_password: "{{ vault_db_password }}" instead of holding the secret?

    The indirection keeps variable names greppable and reviewable, while only the Vault-encrypted file contains the secret.
  2. 2. The app role creates a database user. Which keywords make it run once, on the database server?

    delegate_to runs the task on another host (with the current host's variables); run_once runs it a single time for the whole play.
  3. 3. Why does the playbook have three plays in the order db, app, lb?

    Plays run in the order written. Dependencies flow from the database to the apps to the balancer.
  4. 4. How does the HAProxy template produce different configs for dev and prod without edits?

    Roles read the group membership from whichever inventory is selected with -i, so the same code scales from one server to many.

Summary

  • One repository, one playbook, one set of roles; each environment is an inventory directory with its own group_vars.
  • Group names are the contract between inventory, playbook and roles; roles read groups[...] and hostvars[...] instead of hard-coding addresses.
  • Keep secrets in an encrypted vault.yml and reference them from a readable vars.yml; use Vault IDs to separate environments and no_log to keep output clean.
  • Play order expresses dependencies: database, application, load balancer.
  • delegate_to plus run_once performs a one-time action on another machine; serial protects traffic during rollouts.

Next: Project: Build a Mini Ansible in JavaScript.