Skip to content
Tutorials Center
My learning

Topics

Lesson 12 of 37 ยท Key-Value and Config

The Consul Key-Value Store

Store and read configuration in Consul's key-value store with the CLI and HTTP API: key hierarchies, base64 values, recurse, keys with separators, indexes and size limits.

Besides service discovery, Consul has a small key-value (KV) store. It is strongly consistent (it lives in the Raft log, see the architecture lesson), replicated across servers and available through the CLI, the HTTP API and the web UI. Teams use it for dynamic configuration, feature flags, coordination between processes and small bits of shared state.

It is not a database. Values are limited to 512 KB, there is no querying beyond key prefixes, and every write goes through the leader. Think of it as a shared, highly available configuration directory. The commands below are read-only; the stand-ins model KV behavior, including the index numbers Consul attaches to every key.

The CLI

Terminal
$ consul kv put app/color blue
$ consul kv put app/max_connections 100
$ consul kv get app/color
$ consul kv get -detailed app/color
$ consul kv get -recurse app/
$ consul kv get -keys app/
$ consul kv delete app/color
$ consul kv delete -recurse app/
  • put creates or overwrites a key. Reading from a file or standard input works: consul kv put app/cert @cert.pem or echo hi | consul kv put app/greeting -.
  • get prints the value; -detailed also prints CreateIndex, ModifyIndex, LockIndex, Flags and Session.
  • -recurse works on a whole prefix, -keys lists only key names.
  • Deleting a prefix without -recurse deletes just the exact key.

Keys look like paths, but the store is flat: the slash is only a convention that makes prefix queries convenient. Pick a layout such as apps/<service>/<environment>/<setting>:

TEXT
apps/web/prod/log_level
apps/web/prod/feature_flags/new_checkout
apps/web/staging/log_level
shared/db/host

The HTTP API

Terminal
$ curl --request PUT --data 'blue' http://127.0.0.1:8500/v1/kv/app/color
$ curl http://127.0.0.1:8500/v1/kv/app/color
$ curl "http://127.0.0.1:8500/v1/kv/app/color?raw"
$ curl "http://127.0.0.1:8500/v1/kv/app/?recurse"
$ curl "http://127.0.0.1:8500/v1/kv/app/?keys&separator=/"
$ curl --request DELETE "http://127.0.0.1:8500/v1/kv/app/?recurse"

A normal GET returns a JSON list with objects like this (fields shown, example values invented):

JSON
[{
  "Key": "app/color",
  "Value": "Ymx1ZQ==",
  "Flags": 0,
  "CreateIndex": 101,
  "ModifyIndex": 101,
  "LockIndex": 0
}]

Three details surprise newcomers:

  1. Value is base64. The API cannot return arbitrary bytes in JSON, so you must decode it (Ymx1ZQ== is blue). Add ?raw to get the plain bytes instead.
  2. A missing key is HTTP 404, not an empty value. An empty string is a valid, existing value.
  3. Indexes are cluster-wide counters. CreateIndex is the Raft index when the key was created, ModifyIndex when it last changed. They only increase, and the next two lessons use them for safe updates and for waiting on changes.

?keys lists names only; with separator=/ it behaves like a directory listing, returning one entry per "folder" (app/db/) instead of every nested key. Flags is a 64-bit number your application can use as it likes, for example to mark a value as JSON.

Importing and exporting

Terminal
$ consul kv export app/ > app-config.json
$ consul kv import @app-config.json

Export writes JSON with base64 values, ideal for backups of configuration, copying between environments, and diffing in code review. Never commit exported secrets.

The KV store is not encrypted at rest by default and every holder of a token that can read the key sees the plaintext. Store secrets in a proper secrets manager (such as HashiCorp Vault) and keep only non-secret settings, or references to secrets, in Consul KV. Restrict who can read and write paths with ACLs (chapter 6).

Transactions

Several operations can be applied atomically with PUT /v1/txn. The body lists up to 64 operations, such as set, get, delete, cas and check-index. Either all of them are applied or none: updating a feature flag and its version stamp together is a typical use.

Stand-in 1: a KV store with indexes

This model reproduces the observable behavior: a global index that increments on every write, per-key CreateIndex and ModifyIndex, base64 values and a 404 for missing keys. It is a teaching model, not Consul.

Python Stand-in: KV with indexes

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Stand-in 2: prefix and separator listing

?keys&separator=/ collapses nested keys into folder names. The model below does the same on a list of invented keys.

Python Stand-in: list keys like folders

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Good habits

  • Use a clear hierarchy and consistent names; prefix keys per application and environment.
  • Keep values small. Large blobs slow Raft replication; store them elsewhere and keep a pointer.
  • Write configuration changes with a version or a review process, not by hand-editing in production.
  • Remember that reads are strongly consistent by default, so a heavy polling loop on a hot key adds load; use blocking queries (see watches).

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What is the maximum size of a single KV value?

    Consul limits values to 512 KB because they are replicated through Raft.
  2. 2. A GET on the API returns the Value field as Ymx1ZQ==. What is it?

    KV values are base64 in JSON replies. Use ?raw to receive the bytes directly.
  3. 3. What does GET /v1/kv/does-not-exist return?

    A missing key is a 404. An existing key with an empty value returns 200.
  4. 4. Why should you not store passwords in Consul KV?

    KV is for configuration. A secrets manager such as Vault is designed for secrets.

Summary

  • Consul KV is a small, strongly consistent, replicated configuration store (values up to 512 KB).
  • Use consul kv put/get/delete/export/import or /v1/kv/<key>; keys are flat paths by convention.
  • API values are base64; missing keys give 404; ?raw, ?recurse, ?keys&separator=/ change the shape.
  • CreateIndex and ModifyIndex increase over time and enable safe updates and blocking queries.
  • Do not store secrets; use ACLs to restrict paths.

Next: Check-and-Set, Sessions and Locks