Skip to content
Tutorials Center
My learning

Topics

Lesson 23 of 37 ยท Service Mesh

Consul Traffic Routing, Splitting and Resolution

Shape mesh traffic with Consul config entries: service-router for L7 routing, service-splitter for canary releases and service-resolver for subsets and failover.

Once services talk through sidecar proxies, Consul can do more than allow or deny calls. It can decide where each request goes: send /admin to a different service, release a new version to 5% of users, or fail over to another datacenter when a service is down. These behaviors are called L7 traffic management and are configured with three config entries that always run in a fixed order: router, splitter, resolver.

The product cannot run on this site. The HCL is read-only and the runnable examples are stand-ins that model the same logic.

Prerequisite: a layer 7 protocol

Routing by path or header needs the mesh to understand requests, so the services involved must declare http (or grpc):

HCL
Kind     = "service-defaults"
Name     = "api"
Protocol = "http"

Apply with consul config write api-defaults.hcl. Set the protocol on every service in the chain (router target, splitter targets) before adding rules.

The pipeline

flowchart LR
  R["service-router: match path, header, method"] --> S["service-splitter: weights between subsets"]
  S --> V["service-resolver: subsets, failover, redirect"]
  V --> I["healthy instances"]

Each stage is optional. A request that goes through all three is routed, then split, then resolved to real instances.

Every service has an implicit router, splitter and resolver with defaults (route everything to itself, 100% to itself). You only write the ones you need.

service-resolver: subsets and failover

The resolver defines subsets of a service, which are named groups of instances selected by a filter on metadata, and what to do when instances are unhealthy.

HCL
Kind          = "service-resolver"
Name          = "api"
DefaultSubset = "v1"

Subsets = {
  v1 = { Filter = "Service.Meta.version == v1" }
  v2 = { Filter = "Service.Meta.version == v2" }
}

Failover = {
  "*" = { Datacenters = ["dc2"] }
}

ConnectTimeout = "5s"

For this to work, the api instances must be registered with meta = { version = "v2" }. Failover sends traffic to another datacenter when no healthy local instance exists. A resolver can also Redirect to a different service or datacenter, useful for migrations.

service-splitter: weights and canaries

The splitter divides traffic between subsets or services by weight. Weights are percentages and must sum to 100.

HCL
Kind = "service-splitter"
Name = "api"

Splits = [
  { Weight = 90, ServiceSubset = "v1" },
  { Weight = 10, ServiceSubset = "v2" },
]

A canary rollout is simply a sequence of rewrites: 99/1, then 90/10, 50/50, finally 0/100 followed by removing v1. Each split can also set RequestHeaders or ResponseHeaders modifications. Because the proxy makes the choice for each request, the split is statistical and not sticky: one user may bounce between versions unless you route by header or cookie in a router.

Python Stand-in: weighted traffic split

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The exact counts depend on the random generator, so read the output as "close to 90/10", not as a promise of exact numbers.

service-router: match requests

The router looks at the request and picks a destination. Routes are evaluated in order, first match wins, and an implicit final route sends everything else to the service itself.

HCL
Kind = "service-router"
Name = "web"

Routes = [
  {
    Match {
      HTTP {
        PathPrefix = "/admin"
      }
    }
    Destination {
      Service = "admin"
    }
  },
  {
    Match {
      HTTP {
        Header = [
          { Name = "x-beta", Exact = "1" }
        ]
      }
    }
    Destination {
      Service       = "web"
      ServiceSubset = "beta"
    }
  }
]

Match options include PathExact, PathPrefix, PathRegex, Header, QueryParam and Methods. Destination options include Service, ServiceSubset, PrefixRewrite, NumRetries, RetryOnConnectFailure, RequestTimeout and IdleTimeout.

Python Stand-in: first-match router

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Putting it together

For a safe release of api v2:

  1. Register v2 instances with meta = { version = "v2" } and give them health checks.
  2. Write a resolver with subsets v1 and v2.
  3. Write a splitter with 100/0 (explicit baseline), then ramp up 1%, 10%, 50%.
  4. Watch error rates and latency from the proxies' metrics between steps.
  5. Roll back by writing the splitter back to 100/0, which takes effect in seconds without a deploy.
  6. Retire v1 and delete the splitter and subsets.

Keep entries in version control and use consul config read -kind service-splitter -name api to see what is live. consul config list -kind service-router lists entries of one kind and consul config delete removes one.

Retries, timeouts and limits

Beyond routing, proxies provide resilience settings:

  • Retries and timeouts on router destinations.
  • Circuit breaking through UpstreamConfig.Defaults.Limits in service-defaults: MaxConnections, MaxPendingRequests, MaxConcurrentRequests.
  • Passive health checks with PassiveHealthCheck (Interval, MaxFailures) to eject misbehaving instances.

Use retries carefully: retrying non-idempotent requests (POST that charges a card) can duplicate work.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. In what order does Consul apply the three L7 config entries to a request?

    The router picks a destination, the splitter divides it by weight and the resolver finds the real instances.
  2. 2. What is required of the weights in a service-splitter?

    Weights are percentages of traffic, so the total must be 100.
  3. 3. How are subsets such as v1 and v2 defined?

    The resolver's Subsets use a filter expression like Service.Meta.version == v2.
  4. 4. A router has /admin first and a catch-all second. A request to /admin/x matches both. What happens?

    Routes are evaluated in order and the first match is used.
  5. 5. What must be set before routing on paths or headers?

    The proxy can only inspect requests when the protocol is layer 7.

Summary

  • Router, splitter and resolver form an ordered pipeline for L7 traffic; each is a config entry applied with consul config write.
  • Resolvers define subsets, failover and redirects; splitters divide traffic by weight for canaries; routers match paths, headers and methods.
  • Services need an http or grpc protocol first.
  • Roll out gradually, watch metrics and keep rollback to one config write.

Next: The ACL System