Skip to content
Tutorials Center
My learning

Topics

Lesson 28 of 36 ยท Deployment and HA

Vault Integrated Storage (Raft): Clusters, Quorum and Snapshots

Understand Vault Integrated Storage based on Raft: leader election, quorum math, cluster sizes, retry_join, adding and removing peers, autopilot and snapshots.

Vault needs a durable place to keep its encrypted data. Integrated Storage puts that data on each Vault node's own disk and uses the Raft consensus protocol to keep all nodes in agreement. You need no extra Consul cluster, and one system to run, upgrade and secure. HashiCorp recommends it for most new deployments.

No cluster runs on this site. Commands and HCL are shown for real servers; the JavaScript examples are stand-ins for quorum math and leader election logic.

How Raft works in a nutshell

One node is the leader; the others are followers. All writes go to the leader, which appends them to a log and replicates it. A write is committed once a majority (quorum) of voters has stored it. If the leader fails, followers hold an election and a node with an up-to-date log becomes the new leader. Because a majority must always agree, the cluster cannot split into two independent halves ("split brain").

flowchart LR
    C["Client write"] --> L["Leader"]
    L -->|"replicate"| F1["Follower 1"]
    L -->|"replicate"| F2["Follower 2"]
    F1 -->|"ack"| L
    F2 -->|"ack"| L
    L -->|"committed after majority"| C

A write is acknowledged after most voters have it.

Quorum and cluster sizes

Quorum is floor(n / 2) + 1. The number of failures you can survive is n - quorum.

Voters Quorum Failures tolerated
1 1 0
3 2 1
5 3 2
7 4 3

Use an odd number: 4 nodes tolerate no more failures than 3. Five nodes spread over three availability zones is a common production design; more than seven slows writes with little gain.

JS Stand-in: quorum calculator

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Configuring a node and joining a cluster

HCL
storage "raft" {
  path    = "/opt/vault/data"
  node_id = "vault-1"

  retry_join {
    leader_api_addr = "https://vault-2.example.internal:8200"
  }
  retry_join {
    leader_api_addr = "https://vault-3.example.internal:8200"
  }
}

Initialize one node (vault operator init), unseal it, then start the others; retry_join makes them find the leader and join. Unseal each one (or use auto-unseal). Inspect the cluster:

Terminal
vault operator raft list-peers
vault operator raft autopilot state
vault operator raft remove-peer vault-3

You must give the retry_join addresses TLS trust (leader_ca_cert_file) when using a private CA.

Autopilot

Autopilot automates operations: removing dead servers after a threshold, promoting new nodes to voters only when healthy and up to date (server_stabilization_time), and upgrading nodes in an orderly way (Enterprise). It also reports a cluster health summary that you should alert on.

Snapshots

Raft allows consistent snapshots of the whole data set:

Terminal
vault operator raft snapshot save backup.snap
vault operator raft snapshot restore backup.snap

Snapshots are the foundation of your backup plan; see backup and DR. Storage must be on fast local SSD disks: Raft fsyncs every write, so slow or network-attached disks hurt latency.

JS Stand-in: a simplified leader election

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Practical tips

  • Keep nodes in different failure domains but with low latency between them (same region).
  • Do not run a two-node cluster; it has no fault tolerance.
  • Replace a failed node: remove the peer, add a new one, let it catch up.
  • Take regular snapshots and test restores.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What is the quorum for a 5-node Raft cluster?

    floor(5/2) + 1 = 3, so two nodes can fail.
  2. 2. Why is an even number of voters discouraged?

    4 voters tolerate 1 failure, the same as 3.
  3. 3. Which command lists the nodes in the Raft cluster?

    list-peers prints each node, its address and role.
  4. 4. What does retry_join do?

    It points a starting node at one or more existing members.

Summary

  • Integrated Storage keeps data on each node and replicates with Raft.
  • Quorum is a majority; use 3 or 5 voters.
  • Join with retry_join, manage peers with vault operator raft.
  • Autopilot and snapshots keep the cluster healthy and recoverable.

Next: High Availability and Auto-Unseal