Skip to content
Tutorials Center
My learning

Topics

Lesson 21 of 36 ยท Observability

Istio Access Logs and Envoy Response Flags Explained

Turn on Envoy access logs in Istio, read the default format, filter logs with the Telemetry API and decode response flags such as UH, UF, UO, NR and URX when debugging 503 errors.

Metrics give you totals and traces give you the path of one request. Access logs give you the plain facts of every single request: who called, what was asked, what came back and, crucially, why the proxy answered the way it did. When a request fails with a 503 and the application logs show nothing, the Envoy access log is usually where the answer is.

The product cannot run on this site, so the YAML and kubectl commands are read-only. A Python stand-in parses a log line and decodes the flags, the same thing you will do by eye or with a log pipeline.

Turning access logs on

Access logs are off by default in the default profile. There are two ways to enable them.

Quick, mesh-wide, through mesh configuration:

Terminal
istioctl install --set meshConfig.accessLogFile=/dev/stdout
# JSON output instead of text:
istioctl install --set meshConfig.accessLogFile=/dev/stdout --set meshConfig.accessLogEncoding=JSON

Flexible, per namespace or workload, with the Telemetry API:

YAML
apiVersion: telemetry.istio.io/v1
kind: Telemetry
metadata:
  name: log-errors-only
  namespace: shop
spec:
  accessLogging:
    - providers:
        - name: envoy
      filter:
        expression: "response.code >= 400"

The filter.expression uses a CEL expression, so you can log only errors, or only requests slower than a threshold, and keep log volume (and cost) under control. A provider named envoy writes to the proxy's stdout. You can also define providers that send logs to an OpenTelemetry collector.

Read the logs from the proxy container:

Terminal
kubectl logs deploy/reviews -c istio-proxy --tail=20

The default text format

A line looks like this (shortened):

TEXT
[2026-03-01T10:15:02.114Z] "GET /reviews/1 HTTP/1.1" 503 UH no_healthy_upstream "-" 0 19 0 - "-" "curl/8.5" "5d3f..." "reviews:9080" "-" outbound|9080||reviews.shop.svc.cluster.local - 10.96.12.4:9080 10.1.4.7:51122 - default

The fields, in order: start time, request line (method, path, protocol), response code, response flags, response code details, upstream transport failure reason, bytes received, bytes sent, duration, upstream service time, x-forwarded-for, user agent, request id, authority, upstream host, upstream cluster, upstream local address, downstream local address, downstream remote address, requested server name (SNI) and route name.

The upstream_cluster value has a pattern: outbound|9080|v2|reviews.shop.svc.cluster.local means direction, port, subset, service host. An empty subset between the bars means "no subset".

Response flags: the most useful column

The flag column says why Envoy ended the request. The ones you will meet most often:

Flag Meaning Typical cause
- No flag Normal request
UH No healthy upstream All endpoints ejected or none ready
UF Upstream connection failure Pod not listening, mTLS mismatch
UO Upstream overflow Circuit breaker tripped
NR No route configured Missing VirtualService or wrong host/port
URX Retry limit exceeded All retries failed
UT Upstream request timeout Route timeout too low or slow service
DC Downstream connection terminated Client gave up
UC Upstream connection terminated Server closed the connection
RL Rate limited Local or global rate limit
DI Delay injected Fault injection
FI Fault injected Fault injection

Flags can be combined, for example UF,URX. Remember them: a 503 with UO is a resilience policy at work, while a 503 with NR is a configuration mistake.

Try it: parse a log line

Python Parse an Envoy access log line

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Try it: decode flags and summarize a batch

Python Decode response flags and count failures

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Practical tips

  • Use the x-request-id in the log to find the same request in the upstream's log and in your traces.
  • Log volume is the main cost of access logs. Use the filter expression and keep full logging for debugging sessions.
  • JSON encoding is easier for log pipelines; the text format is easier to read in a terminal.
  • A line in the client's proxy log and a line in the server's proxy log belong to the same request. If only the client has a line, the request never reached the server's proxy.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. A request returns 503 with response flag NR. What is the most likely problem?

    NR means no route configured, usually a missing or mismatched VirtualService or service port.
  2. 2. What does the flag UO mean?

    UO is upstream overflow, produced by connection pool limits in a DestinationRule.
  3. 3. How can you reduce access log volume without losing error visibility?

    The Telemetry API accepts a CEL filter so only matching requests are logged.
  4. 4. In the upstream cluster value outbound|9080|v2|reviews.shop.svc.cluster.local, what is v2?

    The pattern is direction, port, subset, host.

Summary

  • Enable access logs mesh-wide with meshConfig.accessLogFile or selectively with the Telemetry API.
  • Read them with kubectl logs ... -c istio-proxy.
  • The response flag explains why Envoy answered: UH, UF, UO, NR, URX, UT, DC, RL.
  • Filter by status code to control volume and correlate with x-request-id.

Next: Kiali and Grafana dashboards