Skip to content
Tutorials Center
My learning

Topics

Lesson 26 of 37 ยท Clustering and Sessions

Load Balancing Tomcat and Sticky Sessions

Put several Tomcat nodes behind a load balancer: algorithms, jvmRoute sticky sessions, health checks, proxy headers, and zero-downtime draining of a node.

A load balancer is a front door that spreads requests across several Tomcat instances. It gives you more capacity, lets you take one node down for maintenance without an outage, and hides failures from users. The load balancer can be Apache httpd (with mod_proxy_balancer or the older mod_jk), Nginx, HAProxy, or a cloud service. Tomcat itself only needs a few settings to play well with it: a unique jvmRoute, correct proxy headers and a health endpoint.

This lesson covers balancing algorithms, why sticky sessions exist and how they work, health checks and draining, and shows runnable stand-ins of the algorithms. The proxy configuration is read-only, since neither Tomcat nor a proxy runs on this site.

Balancing algorithms

The balancer picks a backend for every request (or every new session). Common strategies:

  • Round robin: take backends in turn. Simple and fair when requests cost about the same.
  • Weighted round robin: bigger machines get more turns.
  • Least connections: send to the node with the fewest active requests; good when request times vary.
  • IP hash: the client address decides the node, a crude form of stickiness.
  • Sticky (session affinity): a given session always goes to the same node while that node is healthy.

Why sticky sessions

Without replication, a session exists only on the node that created it. If the next request goes elsewhere, the user appears logged out. Stickiness solves that by routing every request of a session to the same node. Tomcat helps by appending its jvmRoute to the session id, e.g. JSESSIONID=8F2A91.node2. The balancer reads the suffix and routes to node2.

XML
<!-- conf/server.xml on the second machine -->
<Engine name="Catalina" defaultHost="localhost" jvmRoute="node2">

An Apache httpd balancer using the suffix looks like this:

APACHE
<Proxy "balancer://app">
    BalancerMember "http://10.0.0.11:8080" route=node1 loadfactor=1
    BalancerMember "http://10.0.0.12:8080" route=node2 loadfactor=1
    ProxySet stickysession=JSESSIONID|jsessionid
</Proxy>
ProxyPass        "/" "balancer://app/"
ProxyPassReverse "/" "balancer://app/"

The route must equal the node's jvmRoute. An Nginx setup uses an upstream block with ip_hash or a cookie-based module for the same job:

NGINX
upstream tomcat_pool {
    least_conn;
    server 10.0.0.11:8080 max_fails=3 fail_timeout=10s;
    server 10.0.0.12:8080 max_fails=3 fail_timeout=10s;
}
server {
    listen 80;
    location / {
        proxy_pass http://tomcat_pool;
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

Sticky sessions are a compromise. They keep load uneven (a heavy user stays on one node), and if the node dies, its sessions are lost unless they are replicated (see session replication). Combine stickiness with replication for the smoothest failover, or avoid the need by keeping the app stateless.

Tomcat settings behind a proxy

For the application to generate correct URLs and for security rules to work, Tomcat must know the original scheme, host and client address. Use RemoteIpValve and tell the connector about the proxy:

XML
<Connector port="8080" protocol="HTTP/1.1" proxyName="www.example.com" proxyPort="443" scheme="https" />
<Valve className="org.apache.catalina.valves.RemoteIpValve" internalProxies="10\.0\.0\.\d+"
       remoteIpHeader="X-Forwarded-For" protocolHeader="X-Forwarded-Proto"/>

Otherwise redirects such as response.sendRedirect("/login") may point to http://internal-host:8080, a classic proxy bug.

Health checks and draining

The balancer should only send traffic to nodes that are ready. A health check requests a lightweight URL (for example /health, a servlet that returns 200 when the application can reach its database) every few seconds. After a number of failures the node is removed from rotation, and re-added after successes.

To deploy without downtime, drain a node: tell the balancer to stop sending new sessions to it (in httpd's balancer-manager, set the member to "drain" status), wait until existing sessions finish or expire, then stop and upgrade it. A parallel deployment can also move users between application versions on a single node.

Stand-in: algorithms side by side

This runnable stand-in compares round robin with least-connections for requests of uneven length. It is a model, not a real balancer.

Java Stand-in: round robin vs least connections

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with IO.readln() or a Scanner), it gets the next line from this box.

With this pattern, round robin sends every heavy request to the same node, so that node's queue grows. Picking the least busy node spreads the work.

Stand-in: sticky routing with failover

The second stand-in reads the jvmRoute suffix from the session id, routes to that node, and falls back to another node (creating a new session) if the owner is down:

Java Stand-in: sticky routing

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with IO.readln() or a Scanner), it gets the next line from this box.

Common mistakes

  • Two nodes with the same jvmRoute: stickiness points at the wrong machine.
  • Health checks that only test that the port is open: a hung application still answers TCP.
  • Not forwarding X-Forwarded-Proto, causing redirect loops between HTTP and HTTPS.
  • Using stickiness as a substitute for serializable sessions and replication when node failure must be survived.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What makes sticky sessions work with Tomcat?

    Tomcat appends jvmRoute to JSESSIONID, and the balancer maps the suffix to a member.
  2. 2. Which algorithm suits requests of very different durations?

    Least connections sends new work to the least busy node, so slow requests do not pile up on one node.
  3. 3. What does draining a node mean?

    Draining lets users finish on the old node before it is stopped for maintenance.
  4. 4. Why configure RemoteIpValve behind a proxy?

    Otherwise the application sees the proxy's address and the internal http scheme.

Summary

  • A load balancer adds capacity and resilience; pick an algorithm to match the workload.
  • Sticky sessions rely on a unique jvmRoute per node; replication protects against node loss.
  • Forward the original scheme, host and client address and configure RemoteIpValve.
  • Use real health checks and drain nodes before maintenance.

Next: Monitoring Tomcat with JMX