Skip to content
Tutorials Center
My learning

Topics

Lesson 22 of 36 ยท Observability

Kiali and Grafana for Istio: Graphs, Health and SLOs

Visualize your mesh with Kiali's service graph and health indicators and Grafana's Istio dashboards. Learn what to watch, how to define SLOs, and how burn-rate alerts work.

Raw metrics, traces and logs are powerful but hard to see at a glance. Istio therefore ships integrations with two visual tools: Kiali, a console made specifically for the mesh, and Grafana, with ready-made Istio dashboards. This lesson explains what each shows, how to open them, and how to turn the numbers into service level objectives (SLOs) and alerts.

The tools themselves are not available here, so commands are read-only and the calculators are Python stand-ins you can run.

Installing the add-ons

Istio's release folder contains demo manifests for Prometheus, Grafana, Kiali and Jaeger. They are intended for evaluation: single replica, no persistence, no authentication.

Terminal
kubectl apply -f samples/addons
kubectl rollout status deployment/kiali -n istio-system

istioctl dashboard kiali
istioctl dashboard grafana

istioctl dashboard creates a temporary port-forward and opens your browser. In production you install each tool through its own Helm chart or operator, put it behind authentication and point it at a durable Prometheus.

Kiali: the mesh at a glance

Kiali reads Prometheus metrics, Kubernetes objects and Istio configuration, and draws them together.

  • Graph: nodes are workloads, services or apps; edges are traffic. Edge labels show requests per second, percent of success, or response time. Animated dots show direction of flow, and a padlock icon marks edges that used mutual TLS. Pick a namespace and a time window at the top.
  • Health: every service and workload gets a status (healthy, degraded, failing) based on pod readiness and the error rate of inbound and outbound traffic.
  • Istio Config: lists Gateways, VirtualServices, DestinationRules and more, with a validation icon when it finds a problem such as a VirtualService referencing a host that has no destination or a missing subset. This catches mistakes before they cause 503s.
  • Workloads, Services, Applications: detail pages with metrics, logs, traces and the Envoy configuration of a pod.
  • Wizards: forms that generate traffic-shifting or fault-injection configuration for you.

Reading the graph is a skill. Look for a red edge (errors), a service with no incoming edge (nobody calls it, or traffic bypasses the mesh) and edges without a padlock (plaintext traffic you expected to be encrypted).

Kiali shows what the metrics say. If an edge is missing, it may only mean no request happened in the selected time window, not that the path is broken.

Grafana: Istio dashboards

The Istio add-on provides pre-built dashboards:

Dashboard Use it to
Mesh Dashboard See global request volume, success rate and the slowest services
Service Dashboard Look at one service: client and server request rates, duration percentiles, request sizes
Workload Dashboard Break a service down by workload and version, handy for canary comparisons
Control Plane Dashboard Check istiod health: config pushes, push errors, proxy convergence time, memory
Performance Dashboard See proxy CPU and memory use

For canaries, the Workload dashboard with a version filter lets you compare the success rate and p99 latency of reviews-v1 against reviews-v2 side by side.

From dashboards to SLOs

A service level indicator (SLI) is a measurement, for example "the share of requests that return a non-5xx response within 300 ms". A service level objective (SLO) is the target, for example "99.9 percent over 30 days". The gap between 100 percent and the target is the error budget: 0.1 percent of requests can fail before you break the promise.

Alerts on raw error rate are noisy. Burn-rate alerts compare how fast you are consuming the budget with the pace that would exhaust it exactly at the end of the window. A burn rate of 1 means the budget will last the whole period; a burn rate of 14.4 means a 30-day budget will be gone in about two days.

Try it: error budget and burn rate

Python Error budget and burn rate calculator

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Try it: a simple health classifier

Kiali's exact thresholds are configurable, so this stand-in uses its own labelled rules to show the idea: look at the error ratio of inbound traffic and decide.

Python Classify service health from request counts

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

What to alert on

Good starting alerts for a mesh: a burn-rate alert per critical service; a sudden drop in request rate (a silent outage); p99 latency above the SLO; istiod config-push errors or proxies not converging; certificate expiry close to the end of its life; and an increase in UH, UO or NR response flags.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does a padlock on a Kiali graph edge indicate?

    Kiali marks edges that carried mTLS traffic, helping you find plaintext connections.
  2. 2. Which Grafana dashboard helps compare two versions of a service during a canary?

    The Workload dashboard breaks metrics down by workload and version.
  3. 3. With a 99.9 percent SLO, what is the error budget?

    The budget is 100 percent minus the objective, so 0.1 percent.
  4. 4. Why use burn-rate alerts instead of a plain error-rate threshold?

    Burn rate relates current errors to the SLO so short harmless blips do not page anyone while fast burns do.

Summary

  • Kiali draws the live service graph, validates Istio configuration and shows health and mTLS usage.
  • Grafana's Istio dashboards cover mesh, service, workload, control plane and performance views.
  • The samples/addons manifests are for demos; production needs durable, secured installs.
  • Define SLIs and SLOs, then alert on error-budget burn rate.

Next: Ingress gateway basics