Skip to content
Tutorials Center
My learning

Topics

Lesson 14 of 30 ยท Proxying and Load Balancing

Load Balancing and Health Checks

Spread traffic over several backends with nginx upstream blocks and Apache mod_proxy_balancer: round-robin, least_conn, ip_hash, weights, backups and failure handling.

One backend is a single point of failure and a single ceiling on capacity. Load balancing puts several identical backends behind one entry point and lets the proxy choose which one gets each request. If one server dies, the others keep serving. If traffic grows, you add a server instead of buying a bigger one.

Both nginx and Apache can do this with only their standard modules. In nginx you declare a named pool with an upstream block and point proxy_pass at it. In Apache you use mod_proxy_balancer together with a load-balancing algorithm module. In this lesson you learn the selection methods, how weights and backups work, how the proxy notices a dead server, and when you need sticky sessions.

You should already know Reverse Proxy Basics, because everything here builds on proxy_pass.

How the proxy picks a server

Round-robin Take turns. The default.
least_conn Fewest active connections wins.
ip_hash Same client, same server.
Weights Bigger servers take more turns.
1 / 2

An upstream pool in nginx

NGINX
upstream app_pool {
    server 10.0.0.11:3000;
    server 10.0.0.12:3000;
    server 10.0.0.13:3000 weight=2;
    server 10.0.0.99:3000 backup;
    keepalive 16;
}

server {
    listen 80;
    location / {
        proxy_pass http://app_pool;
        proxy_http_version 1.1;
        proxy_set_header Connection "";
    }
}

The upstream block lives in the http context, next to your server blocks, and proxy_pass uses its name instead of an address. Note that proxy_pass http://app_pool; has no URI part, so the trailing-slash rule from the previous lesson applies unchanged. The keepalive line keeps up to 16 idle connections to the pool open for reuse, which needs HTTP/1.1 and an empty Connection header as shown.

Selection methods

  • Round-robin is the default: server 1, server 2, server 3, back to 1. With weight it becomes weighted round-robin. nginx uses a smooth variant that spreads a heavy server's turns out instead of sending several in a row.
  • least_conn; sends each request to the server with the fewest active connections (weights are taken into account). It beats round-robin when requests vary a lot in duration.
  • ip_hash; hashes the client address (the first three octets of an IPv4 address), so one client keeps reaching the same server. It is a cheap form of stickiness, but clients behind one big NAT all land on one server, and changing the pool reshuffles many clients.
  • hash $request_uri consistent; hashes any key you like. The consistent flag limits reshuffling when servers come and go, which suits caches.
  • random two least_conn; picks two random servers and then the less busy one. It works well when several balancers share the same pool.

Only one method goes in a block, and it must appear before the server lines.

Weights and backups

weight=2 means "twice the share of a weight=1 server". Use it when machines have different sizes. A server marked backup receives traffic only when all the non-backup servers are unavailable, which is useful for a maintenance page or a smaller standby. A server marked down is taken out manually, and ip_hash pools use it to keep the hashing stable.

Here is the selection logic, written out in JavaScript. The weighted round-robin follows nginx's published smooth algorithm. The ip_hash function is not nginx's real hash: it is a simple stand-in with the same property (same client, same server). Edit the weights and run it again.

JS Weighted round-robin, least_conn and ip_hash

Press Run (or Ctrl + Enter) to see the console output here.

No output.

In the output, the weighted sequence interleaves c with the others instead of giving it two turns in a row. Under least_conn, the slow server c gets 3 of the 24 requests instead of the 8 that round-robin would give it, because it is still busy when the next request arrives (a gets the most because ties go to the first server). And when the pool shrinks from three servers to two, several clients switch servers, which is the weakness of hash-based stickiness.

Passive health checks

Open source nginx has no built-in active probing. Instead it watches real traffic: each server line takes

  • max_fails=N (default 1): how many failed attempts within fail_timeout mark the server unavailable;
  • fail_timeout=T (default 10 seconds): both the window in which failures are counted and how long the server stays out before nginx tries it again.
NGINX
upstream app_pool {
    server 10.0.0.11:3000 max_fails=3 fail_timeout=30s;
    server 10.0.0.12:3000 max_fails=3 fail_timeout=30s;
    server 10.0.0.99:3000 backup;
}

location / {
    proxy_pass http://app_pool;
    proxy_next_upstream error timeout http_502 http_503;
    proxy_next_upstream_tries 2;
    proxy_connect_timeout 2s;
}

What counts as a failure is decided by proxy_next_upstream: by default connection errors and timeouts. HTTP 502, 503 and 504 count only if you list them. When an attempt fails, nginx retries the request on the next server, so the user often sees nothing. Two warnings:

  1. A short proxy_connect_timeout is what makes a dead machine fail fast. With the 60 second default, the first unlucky users wait a full minute.
  2. Retrying is only safe for requests that can be repeated. nginx does not retry non-idempotent requests such as POST after the request was sent, unless you add non_idempotent. That is a deliberate protection against charging a card twice.

The commercial NGINX Plus adds active health checks that probe a URL on a timer. With open source, you either rely on passive checks or run an external checker that edits the config and reloads (see Reloads, Zero-Downtime Deploys and Containers).

The next stand-in simulates the passive logic: a clock that ticks once per request, a dead server, a max_fails limit, a fail_timeout quiet period and a backup. It is a model of the behavior described above, not nginx's code.

JS A balancer with a failing backend

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Read the log from the top. At first, requests that land on web2 fail and are retried on the next server, so every request is still answered. After two failures, web2 is skipped for six ticks, and the extra hop disappears. When the quiet period ends, one request probes it again, fails, and the cycle repeats. The spare backup is never used, because at least one primary is always available. Make all three primaries alive: false and it takes over.

The same idea in Apache

Apache balances with mod_proxy_balancer plus one scheduler module such as mod_lbmethod_byrequests (the default, weighted request counting), mod_lbmethod_bytraffic or mod_lbmethod_bybusyness.

APACHE
<Proxy "balancer://app_pool">
    BalancerMember "http://10.0.0.11:3000" loadfactor=1 retry=30
    BalancerMember "http://10.0.0.12:3000" loadfactor=1 retry=30
    BalancerMember "http://10.0.0.13:3000" loadfactor=2 retry=30
    BalancerMember "http://10.0.0.99:3000" status=+H
    ProxySet lbmethod=byrequests
</Proxy>

ProxyPass        "/" "balancer://app_pool/"
ProxyPassReverse "/" "balancer://app_pool/"

loadfactor is the weight, status=+H makes a member a hot standby (the counterpart of backup), and retry is how many seconds a failed member is left alone before it is tried again, much like fail_timeout. Needed modules: proxy, proxy_http, proxy_balancer and lbmethod_byrequests. Apache has an optional status page (balancer-manager handler) for inspecting members; protect it with access rules or leave it off in production.

Sticky sessions

If your app keeps session data in the memory of one process, a user bouncing between servers loses their login. The right fix is to move sessions into a shared store such as a database or Redis, so any server can serve any user. When that is not possible, make the balancer sticky.

  • nginx open source: ip_hash or hash $cookie_sessionid consistent;. Cookie-inserting stickiness (sticky cookie) belongs to NGINX Plus.
  • Apache: balancer stickiness by cookie is built in:
APACHE
Header add Set-Cookie "ROUTEID=.%{BALANCER_WORKER_ROUTE}e; path=/" env=BALANCER_ROUTE_CHANGED
<Proxy "balancer://app_pool">
    BalancerMember "http://10.0.0.11:3000" route=1
    BalancerMember "http://10.0.0.12:3000" route=2
    ProxySet stickysession=ROUTEID
</Proxy>

Sticky routing has a cost: load is no longer even, and when a server dies, its users lose their sessions anyway.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. A request mix includes both 20 ms and 20 s requests. Which nginx method adapts best?

    least_conn looks at active connections, so a server stuck on slow requests receives fewer new ones. Round-robin ignores how busy a server is.
  2. 2. What does a server marked backup do?

    Backups sit idle until the primary servers are all marked unavailable, then they take over.
  3. 3. In open source nginx, how does the proxy learn that a backend is unhealthy?

    Open source nginx has passive checks only. Active probing is a feature of NGINX Plus or an external tool.
  4. 4. Why is it risky to retry a failed POST on another backend automatically?

    A failure after the request was sent does not prove it was not executed. That is why nginx does not retry non-idempotent requests unless told to.
  5. 5. What is the better fix for lost logins when users move between backends?

    Shared session storage removes the problem. Stickiness is a workaround that still loses sessions when a server dies.

Summary

  • An upstream block (nginx) or a balancer:// pool with BalancerMember lines (Apache) names several backends, and proxy_pass/ProxyPass points at the pool.
  • Round-robin is the default; weight adds capacity-aware shares, least_conn adapts to uneven request times, ip_hash and hash ... consistent give stickiness.
  • Passive checks use max_fails and fail_timeout; set proxy_next_upstream and a short proxy_connect_timeout so failures are quick and invisible.
  • backup servers and status=+H members wait until the primaries fail.
  • Prefer shared session storage over sticky sessions.

Next: keep long-lived connections working through the proxy in WebSockets, Streaming and Long Requests.