Skip to content
Tutorials Center
My learning

Topics

Lesson 11 of 37 ยท Service Discovery

Prepared Queries and Failover

Use Consul prepared queries to save a service lookup with tags, health rules, nearest-first sorting and datacenter failover, then call it from DNS or HTTP.

Plain lookups answer "where is web in this datacenter?". Sometimes you want a smarter question saved under a name: "the nearest healthy db with the replica tag, and if there is none here, try the closest other datacenters". Consul calls these saved lookups prepared queries.

A prepared query is created once (through the HTTP API) and then executed by name from DNS (<name>.query.consul) or HTTP. The API calls are read-only on this site; the stand-ins model failover and name templates.

Creating a query

A prepared query is a JSON document sent to POST /v1/query:

Terminal
$ curl --request POST http://127.0.0.1:8500/v1/query --data '{
  "Name": "db-replica",
  "Service": {
    "Service": "db",
    "Tags": ["replica"],
    "OnlyPassing": true,
    "Near": "_agent",
    "Failover": {
      "NearestN": 2,
      "Datacenters": ["dc3"]
    }
  },
  "DNS": { "TTL": "10s" }
}'

The reply contains the query's ID. Fields:

Field Meaning
Name Name used in DNS and in /v1/query/<name>/execute
Service.Service The service to look up
Service.Tags Required tags; prefix a tag with ! to exclude it
Service.OnlyPassing Hide instances with warning checks
Service.Near Sort by distance to a node, or _agent for the agent asking
Service.NodeMeta Filter by node metadata
Service.Failover.NearestN Try the N nearest other datacenters if there are no healthy results locally
Service.Failover.Datacenters Explicit fallback list, tried in order after the nearest ones
DNS.TTL TTL for DNS answers of this query

Executing a query

Terminal
$ curl http://127.0.0.1:8500/v1/query/db-replica/execute
$ dig @127.0.0.1 -p 8600 db-replica.query.consul
$ dig @127.0.0.1 -p 8600 db-replica.query.consul SRV

The HTTP reply lists Nodes (each with Node, Service and Checks), plus the Datacenter that finally answered, a Failovers count and the DNS settings. Clients that need to know whether they were served from a remote datacenter (for latency or compliance reasons) can read those fields.

Other management calls: GET /v1/query lists queries, PUT /v1/query/<id> updates one and DELETE /v1/query/<id> removes it. GET /v1/query/<id>/explain shows how a templated query would resolve.

How failover works

flowchart TD
  Q["Execute db-replica"] --> L{"Healthy results in local dc?"}
  L -->|yes| R1["Return local results"]
  L -->|no| N{"Try NearestN datacenters by RTT"}
  N -->|found| R2["Return results from that dc"]
  N -->|none| D{"Try listed Datacenters in order"}
  D -->|found| R3["Return results from that dc"]
  D -->|none| E["Return empty result"]

Failover only triggers when the local datacenter has no usable results.

Nearest datacenters are decided by network coordinates measured over the WAN gossip pool, so failover prefers the closest remote site. This requires datacenter federation, covered in the multi-datacenter chapter of the course; in a single datacenter a query behaves like a saved filter.

Templates: one query for many services

A template makes a single query definition serve many names. Match a prefix and substitute the rest into the service name:

Terminal
$ curl --request POST http://127.0.0.1:8500/v1/query --data '{
  "Name": "",
  "Template": { "Type": "name_prefix_match" },
  "Service": { "Service": "${name.full}", "OnlyPassing": true }
}'

With an empty name (a catch-all), executing payments resolves to the service payments, and search resolves to search, both with OnlyPassing. Templates can also use ${name.prefix}, ${name.suffix} and regular-expression captures (a Regexp field in the Template block) for custom mapping, for example sending geo-eu-db to the service db with the tag eu.

Prepared queries versus alternatives

  • Prepared queries are catalog-level and work for plain (non-mesh) services over DNS.
  • Inside a service mesh, failover and subsets are expressed as config entries (the service-resolver kind) and enforced by the proxies. Use those for new mesh designs.
  • For plain HTTP clients, a health query with filter, near and your own retry loop is often enough.

Queries are subject to ACLs. Creating one requires query:write on its name, and executing one requires read access to the target service. A query with a blank name and a template needs a broad rule, so grant it sparingly.

Stand-in 1: failover order

This model gives each datacenter a list of healthy instances and a distance in milliseconds, then follows the rule "local first, then the N nearest, then the explicit list". Invented data, no randomness.

Python Stand-in: query failover

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Stand-in 2: template name mapping

A model of templated names with a prefix and a regular expression. It shows how one query can serve names like geo-eu-db.

Python Stand-in: name templates

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. How do you call a prepared query named db-replica through DNS?

    Prepared queries use the .query.consul form.
  2. 2. When does the Failover setting take effect?

    Local results always win; failover is a fallback.
  3. 3. What does a ! prefix on a tag in Service.Tags do?

    A tag such as !canary filters out instances tagged canary.
  4. 4. What is the purpose of a query template?

    Templates map names such as payments or geo-eu-db onto the right service and tags.

Summary

  • A prepared query is a saved, named lookup with tags, health rules, sorting and failover.
  • Create it with POST /v1/query; run it with /v1/query/<name>/execute or <name>.query.consul.
  • NearestN and Datacenters define the fallback order across datacenters.
  • Templates let one query serve many names.
  • For service-mesh failover use service-resolver config entries instead.

Next: The Key-Value Store