Skip to content
Tutorials Center
My learning

Topics

Lesson 29 of 36 ยท Resilience and Policies

Rate Limiting in Istio: Local and Global Limits

Add rate limits to your mesh: the token-bucket algorithm, Envoy's local rate limit filter configured with EnvoyFilter, the global rate limit service with Redis, descriptors and 429 responses.

A single noisy client, a buggy retry loop or a sudden traffic spike can take down a service that is otherwise healthy. Rate limiting puts a ceiling on how many requests are accepted in a time window and rejects the rest quickly with HTTP 429 Too Many Requests. Istio does not have a dedicated high-level rate-limit resource, but its Envoy proxies include two rate-limit filters you can enable: a local one and a global one.

The product cannot run on this site, so configuration is read-only and the Python stand-ins let you play with the algorithm.

Token bucket: the idea behind local limits

Envoy's local rate limiter uses a token bucket. Imagine a bucket holding up to max_tokens tokens. Every fill_interval it receives tokens_per_fill new tokens (never exceeding the maximum). Each request must take one token; if the bucket is empty, the request is rejected with 429. The maximum allows short bursts, and the refill rate defines the sustained rate.

For example, max_tokens: 100, refilling 10 tokens every second: a burst of 100 requests is fine, then about 10 per second continue.

Local rate limiting

"Local" means each proxy enforces its own bucket, with no network call. It is cheap and fast, but the limit is per pod: ten replicas of a service at 10 per second each allow up to 100 per second in total. Use it for protection (a safety valve), not for exact quotas.

It is configured with an EnvoyFilter, Istio's escape hatch for patching proxy configuration:

YAML
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: local-ratelimit
  namespace: shop
spec:
  workloadSelector:
    labels:
      app: reviews
  configPatches:
    - applyTo: HTTP_FILTER
      match:
        context: SIDECAR_INBOUND
        listener:
          filterChain:
            filter:
              name: envoy.filters.network.http_connection_manager
      patch:
        operation: INSERT_BEFORE
        value:
          name: envoy.filters.http.local_ratelimit
          typed_config:
            "@type": type.googleapis.com/udpa.type.v1.TypedStruct
            type_url: type.googleapis.com/envoy.extensions.filters.http.local_ratelimit.v3.LocalRateLimit
            value:
              stat_prefix: http_local_rate_limiter
              token_bucket:
                max_tokens: 100
                tokens_per_fill: 10
                fill_interval: 1s
              filter_enabled:
                runtime_key: local_rate_limit_enabled
                default_value: { numerator: 100, denominator: HUNDRED }
              filter_enforced:
                runtime_key: local_rate_limit_enforced
                default_value: { numerator: 100, denominator: HUNDRED }
              response_headers_to_add:
                - append_action: OVERWRITE_IF_EXISTS_OR_ADD
                  header: { key: x-local-rate-limit, value: "true" }

Rejected requests get a 429 with the header you added, and show the response flag RL in the access log. The filter_enabled and filter_enforced fields let you run in shadow mode (enabled but not enforced) to see what would be blocked before turning it on.

EnvoyFilter is powerful and fragile: its patch syntax depends on Envoy's internal structure and can break when you upgrade Istio. Keep these resources in version control, label them with the Istio version they were tested on and re-test them in every upgrade.

Global rate limiting

For exact, mesh-wide quotas (for example "100 requests per minute per API key across all replicas") the proxies must share state. The global rate limiter does this through an external rate limit service (Envoy's reference implementation is envoyproxy/ratelimit) backed by Redis. On each request the proxy sends a descriptor to the service, such as ("header_match", "path=/api") or ("remote_address", "203.0.113.9"), and the service answers allow or deny based on its configured limits.

The pieces are: a Deployment and Service for the rate limit service plus Redis; a ConfigMap with limit rules; an EnvoyFilter that adds the rate-limit HTTP filter and an EnvoyFilter that defines rate_limits actions on a route (which request attributes become descriptors). The extra network hop adds a little latency, and you must decide whether to fail open (allow requests if the rate limit service is down, the usual choice) or fail closed.

Rule of thumb: use local limits as the default shield at every service and global limits only where exact per-client quotas are a product requirement, typically at the ingress gateway.

Where to apply a limit

  • At the ingress gateway to protect the whole mesh from external abuse and to enforce per-client quotas.
  • On inbound sidecars of a fragile service to protect it from internal callers.
  • On outbound to protect an external API with a known quota (you do not want to be banned).
  • Combine with circuit breaking: rate limits protect against excess demand, circuit breakers react to unhealthy behavior.

Try it: simulate a token bucket

Python Token bucket simulation

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Try it: per-pod versus global limits

Python Why local limits are per pod

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What HTTP status does a rate-limited request receive by default?

    Rate limit filters answer 429 Too Many Requests.
  2. 2. Why is a local rate limit 'per pod'?

    Local limits need no network call, so every proxy counts independently.
  3. 3. When do you need the global rate limit service?

    Only shared state through the rate limit service can enforce a mesh-wide count.
  4. 4. What is the purpose of running a rate limit in shadow mode?

    filter_enabled without filter_enforced records the effect without rejecting traffic.

Summary

  • Rate limiting answers with 429 and the RL flag; token buckets allow bursts and a sustained rate.
  • Local limits are cheap but per pod; global limits need a rate limit service plus Redis.
  • Both are enabled with EnvoyFilter resources: version them and re-test on upgrades.
  • Apply limits at the gateway, on fragile inbound services and on outbound calls to quota-limited APIs.

Next: Sidecar scoping and proxy resources