Skip to content
Tutorials Center
My learning

Topics

Lesson 13 of 65 ยท Load Balancing and Traffic

Load Balancers and Their Types

Learn what a load balancer does, the differences between Layer 4 and Layer 7, hardware, software and cloud options, DNS and global balancing, and sticky sessions.

Once you run more than one server, someone has to decide which server gets each request. That someone is the load balancer: a component that sits in front of a group of servers (a pool or target group), accepts traffic on one address and distributes it across the group. Clients see one endpoint; behind it can be two servers or two thousand.

A load balancer delivers three things at once. It raises capacity by spreading work, it improves availability by skipping dead servers, and it makes maintenance painless, because you can drain one server, upgrade it and put it back without users noticing.

flowchart LR
  C["Clients"] --> LB["Load balancer<br/>one public address"]
  LB --> S1["Server 1"]
  LB --> S2["Server 2"]
  LB --> S3["Server 3"]

Figure: clients know one address; the balancer chooses the server. How it chooses is the subject of the next lesson.

Layer 4 and Layer 7

Load balancers are classified by the layer of the network stack they look at.

Layer 4 (transport) load balancers

A Layer 4 balancer sees only IP addresses and TCP or UDP ports. It picks a server per connection and forwards the packets without reading the contents. Because it does no parsing, it is extremely fast, handles any protocol (databases, game traffic, TLS passthrough) and needs little CPU. It cannot make decisions based on URLs, headers or cookies.

Layer 7 (application) load balancers

A Layer 7 balancer understands the application protocol, usually HTTP. It terminates the connection, reads the request, and decides based on the host name, URL path, headers, cookies or method. That unlocks richer behavior:

  • Route /api/* to API servers and /images/* to media servers.
  • Terminate TLS and re-encrypt, compress and cache.
  • Send Host: shop.example.com to a different pool than blog.example.com.
  • Insert headers (X-Forwarded-For), rewrite URLs, redirect HTTP to HTTPS.
  • Apply rate limits and block bad requests.
  • Do canary releases: 5% of requests to a new version.

The price is more CPU and a little more latency per request. A Layer 7 balancer is essentially a reverse proxy with health checks and balancing logic.

Layer 4 Layer 7
Looks at IP, port URL, headers, cookies, body
Decides per Connection Request
Speed Very high High, more CPU
Protocols Any TCP/UDP HTTP, gRPC, WebSocket
TLS Passthrough (or terminate) Terminates and inspects
Routing by path or host No Yes

Layer 4 or Layer 7?

Layer 4 Fast and simple. Sees addresses and ports only.
Layer 7 Reads the request. Routes by path, host, cookie.
Often both L4 at the edge, L7 for smart routing behind it.

Pick the simplest layer that meets the routing need.

1 / 2

Hardware, software and cloud balancers

  • Hardware load balancers are dedicated appliances with specialized chips. Very high throughput and long track record, but expensive, with a fixed ceiling and slower to change.
  • Software load balancers run on commodity servers or containers: Nginx, HAProxy, Envoy, Traefik. Flexible, scriptable, cheap, and scale out by adding instances.
  • Cloud load balancers are managed services (application, network and gateway balancers in the major clouds). You configure rules; the provider handles capacity and redundancy.

Global server load balancing (GSLB)

When your servers are in several regions, you need to choose a region before you choose a server. Common techniques:

  • DNS-based routing: the DNS answer depends on who asks: the nearest region, the healthiest one, or weighted by capacity. Simple, but caching of DNS answers delays changes.
  • Anycast: many locations announce the same IP address and the network delivers each packet to the nearest one. Used by CDNs and DNS providers.
  • Latency or geography rules in a traffic manager, with health checks and automatic failover to another region.
flowchart TB
  U["User in Europe"] --> D["DNS / traffic manager"]
  D -->|"nearest healthy"| E["EU region balancer"]
  D -.->|"failover"| A["US region balancer"]
  E --> S1["EU servers"]
  A --> S2["US servers"]

Figure: a global layer picks the region; a regional balancer picks the server.

Stateless and stateful balancing

A stateless load balancing decision uses only information in the packet or request itself (for example, a hash of the client IP, or the URL path). No memory is needed, so many balancer instances can make identical decisions with no coordination. A stateful balancer remembers which backend each connection or session was assigned (a connection table, a session cookie map) and keeps sending the same client there.

Stateful choices are needed for sticky sessions (see stateless systems): a user stays on the same server because that server holds their session. Prefer stateless app servers with a shared session store, and use stickiness only as a last resort.

Ways to implement stickiness:

  • Source IP hash: same client IP, same server. Breaks behind shared NATs (an office looks like one client) and when the pool changes.
  • Cookie-based affinity: the balancer sets a cookie naming the server.
  • Consistent hashing on a key, which minimizes reshuffling when servers come and go (consistent hashing).

What a load balancer must also do

  • Health checks: probe each server and stop sending traffic to unhealthy ones (health checks and failover).
  • Connection draining: let in-flight requests finish before a server is removed.
  • TLS termination (optionally re-encrypting to backends).
  • Rate limiting and DDoS absorption at the edge (rate limiting).
  • Observability: per-backend latency, errors and traffic.
  • Its own redundancy: a single balancer is a single point of failure. Run a pair or a cluster (active-passive with a floating IP, or active-active with anycast or DNS).

Simulating the effect of balancing

The next simulation shows the point of a balancer: with one server a load spike creates a backlog; with a balanced pool the same spike is absorbed. It also shows a server being drained for maintenance.

Python One server vs a balanced pool

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

One server builds a large backlog and never fully recovers within the window. Three servers absorb the spike with no backlog. Draining one server during the spike still works: the two live servers build a small backlog (40 requests each) that clears by the end, because they run much closer to their limits: a reminder to keep the N+1 headroom from the capacity lesson.

Routing rules at Layer 7

A Layer 7 balancer is configured with rules such as "if path starts with /api then pool A; if host is static.example.com then pool B; else default pool". This simulation evaluates rules in order, the way real balancers do, including a weighted canary split.

Python Layer 7 routing rules with a canary

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

About ten percent of the /about traffic lands on web-pool-v2. If its error rate or latency worsens, you remove the rule and everyone returns to the stable pool, which is far safer than switching everyone at once.

Choosing a balancer

  • Need maximum raw speed, non-HTTP protocols, or TLS passthrough: Layer 4.
  • Need path or host routing, header handling, canary releases, caching: Layer 7.
  • Multi-region users: add a global layer (DNS, anycast or a CDN).
  • Small team: prefer a managed cloud balancer; large custom needs: software balancers you operate.
  • Always plan the balancer's own redundancy.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What can a Layer 7 load balancer do that a Layer 4 balancer cannot?

    Layer 7 reads the application request, so it can decide using HTTP details.
  2. 2. Why is a single load balancer a problem?

    If the balancer fails, no traffic reaches any server, so it must be made redundant.
  3. 3. Why is source-IP hashing a fragile way to keep sessions sticky?

    An office behind one address overloads one server, and pool changes reshuffle clients.
  4. 4. What does connection draining do?

    New requests stop going to the server while existing ones complete, so maintenance is invisible to users.

Summary

  • A load balancer fronts a pool of servers to add capacity, availability and easy maintenance.
  • Layer 4 balancers are fast and protocol-agnostic; Layer 7 balancers route by request content and add TLS, caching and canaries.
  • Options: hardware appliances, software (Nginx, HAProxy, Envoy) or managed cloud services; global balancing picks the region first.
  • Prefer stateless servers; use sticky sessions only when needed, and balance the balancer's own redundancy.
  • Health checks, draining and observability turn a router into a reliable front line.

Next: Load Balancing Algorithms