Skip to content
Tutorials Center
My learning

Topics

Lesson 5 of 36 ยท Service Mesh Concepts

The Problems a Service Mesh Solves

See the network problems every microservice system meets (latency, failures, identity, visibility) and how a mesh addresses each, with a failure simulator.

A service mesh is easiest to understand as a list of problems. Each one appears the moment you split a program into services that talk over a network. This lesson walks through them in the order teams usually hit them, and for each one says what you would do without a mesh and what a mesh changes.

The network is not a function call

Inside one process, calling a function is fast and almost never fails. Across a network:

  • The call can be slow (a busy server, a retransmit, a cold cache).
  • The call can fail (connection refused, reset, timeout, a crashed pod).
  • The call can half-succeed (the server did the work but the reply was lost).
  • The callee may be somewhere else next time (pods are replaced, IPs change).
  • Anyone on the network might listen or pretend to be the callee.

These are the classic "fallacies of distributed computing": the network is not reliable, latency is not zero, bandwidth is not infinite, the network is not secure, topology changes. A mesh does not remove the fallacies; it gives you one place to handle them.

Four problems, four answers

Reliability Timeouts, retries, circuit breaking.
Security Mutual TLS and call policy.
Visibility Metrics, logs and traces.
Control Canaries and traffic shifting.
1 / 2

Reliability

Without a mesh each client library implements its own retries and timeouts, often wrongly: a retry loop with no limit can turn a small outage into a flood (a retry storm). With a mesh the proxy applies one policy per route: "time out after 2 s, retry twice on connect failure". Chapter 7 covers it. The policy is data, so you can change it during an incident without a deployment.

Security

Inside a cluster, plain Kubernetes lets any pod talk to any pod by default, and traffic is unencrypted. Network policies filter by IP and port, but IPs are ephemeral and say nothing about who is calling. A mesh gives each workload a cryptographic identity and encrypts every hop with mutual TLS, so a rule can say "only the web service account may call catalog with GET". See mTLS and Peer Authentication and Authorization Policies.

Visibility

When a page loads in 3 seconds, which of the eight services behind it was slow? Without shared instrumentation you add metrics code to each service, in each language. A mesh proxy sees every request, so it can emit the same golden signals for all services:

  • Latency: how long requests take (usually as percentiles).
  • Traffic: requests per second.
  • Errors: the share of failing requests.
  • Saturation: how full the resources are (the proxy sees only part of this).

Control

Deploying a new version to everyone at once is a gamble. Kubernetes splits traffic only in proportion to pod counts: to send 1% to the new version you would need 100 pods. A mesh splits by weight, header or path, independent of replica counts. See Weighted Routing and Canary Releases.

What a mesh does not solve

  • It does not fix a slow database query or a bad algorithm.
  • It cannot make a non-idempotent operation safe to retry (retrying a payment call can charge twice).
  • It does not replace application-level authentication of end users, though it can validate tokens.
  • It does not remove the need to design for failure; it only makes the policy uniform.

A failure simulator

This stand-in calls a service whose failure rate you set, and compares callers with no retries against callers that retry. The random generator is seeded, so the numbers are the same every time. It models the idea of retries, not Envoy's behaviour.

Python Stand-in: how much do retries help?

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Retries lift the success rate, but they multiply load on the callee. The second stand-in counts that cost.

Python Stand-in: the cost of retrying everything

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

When the callee is already failing 90% of the time, three attempts send it about 2.7 times the traffic, which is why policies also include retry budgets and circuit breakers (see Weighted Routing and Canary Releases for the routing side and chapter 7 for resilience).

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why is a plain Kubernetes Service weak for a 1% canary?

    Kubernetes Services balance across ready endpoints; the split follows the number of pods, not a weight.
  2. 2. What do the golden signals describe?

    They are the four signals commonly used to judge service health.
  3. 3. Which problem does a mesh NOT solve?

    Retrying a non-idempotent operation can repeat its side effects; only application design can fix that.
  4. 4. What is a retry storm?

    Unlimited retries multiply load on a struggling service and can turn a partial outage into a full one.

Summary

  • Networks are slow, unreliable, observable by others and constantly changing.
  • A mesh addresses reliability, security, visibility and control in one place.
  • Policies are data, so you can change them without redeploying applications.
  • Retries help but cost load; they need limits.
  • A mesh does not fix application bugs or make unsafe operations safe.

Next: Mesh, Libraries and Gateways Compared