Skip to content
Tutorials Center
My learning

Topics

Lesson 28 of 37 ยท Monitoring and Tuning

Thread Pools and Connector Tuning in Tomcat

Understand how Tomcat's NIO connector handles requests, what maxThreads, acceptCount and maxConnections mean, how keep-alive and timeouts work, and how to size a pool.

When a request arrives at Tomcat, a connector accepts the TCP connection and hands the work to a thread from a pool. Everything about throughput, latency under load and how Tomcat fails when overloaded comes down to a few connector settings. Many outages are not bugs but a pool that is too small, a queue that hides trouble, or timeouts that let slow clients hold threads forever.

This lesson explains the request path, the key attributes with sensible starting values, and a way to reason about pool size. Tomcat cannot run here, so settings are read-only. Two runnable Java stand-ins use a real ThreadPoolExecutor to model what happens when requests exceed capacity.

The request path in the NIO connector

The default Http11NioProtocol uses a small number of threads for network I/O and a larger pool for request processing:

flowchart LR
  C["Clients"] --> A["Acceptor thread"]
  A --> P["Poller (NIO selector)"]
  P --> W["Worker pool (maxThreads)"]
  W --> S["Servlet code"]
  A -. "backlog full" .-> R["Connection refused"]

The acceptor takes new connections, the poller watches idle ones, and a worker thread runs the servlet only while a request is being processed.

Because idle keep-alive connections wait in the poller rather than holding a thread, 10,000 open connections do not need 10,000 threads. A thread is busy only while your code runs, which is why slow code (database calls, remote APIs) is the main reason pools run out.

The key attributes

XML
<Connector port="8080" protocol="HTTP/1.1"
           maxThreads="200" minSpareThreads="10"
           maxConnections="8192" acceptCount="100"
           connectionTimeout="20000" keepAliveTimeout="15000" maxKeepAliveRequests="100"
           maxHttpHeaderSize="8192" maxPostSize="2097152"
           compression="on" compressibleMimeType="text/html,text/css,application/json,application/javascript"
           redirectPort="8443"/>
Attribute Meaning Default
maxThreads most request-processing threads running at once 200
minSpareThreads threads kept ready even when idle 10
maxConnections open connections Tomcat accepts and tracks 8192 (NIO)
acceptCount OS queue length for connections once maxConnections is reached 100
connectionTimeout ms to wait for the request line after accept 60000 (the shipped server.xml sets 20000)
keepAliveTimeout ms an idle keep-alive connection stays open same as connectionTimeout
maxKeepAliveRequests requests per connection before closing 100

Order of limits as load grows: first up to maxThreads requests run in parallel; further accepted connections wait in the executor queue (their requests are queued until a worker is free); once maxConnections is reached new connections wait in the OS backlog of size acceptCount; when that is full, clients get "connection refused". Users feel the queue as rising latency long before they see errors.

You can share one pool between connectors with an <Executor>, which also lets you name the threads and set maxQueueSize:

XML
<Executor name="tomcatThreadPool" namePrefix="catalina-exec-" maxThreads="300" minSpareThreads="25"/>
<Connector executor="tomcatThreadPool" port="8080" protocol="HTTP/1.1"/>

Sizing the pool

Little's Law gives a starting point: the number of requests in flight equals the arrival rate multiplied by the average time each takes. If you serve 400 requests per second and each takes 150 ms, about 60 are in flight on average, so allow headroom for peaks, around 120 to 200 threads. More threads is not better. Each thread costs stack memory (about 1 MB by default) and CPU context switches, and if the real bottleneck is a database pool of 20 connections, 500 threads only add queueing in a worse place. Match maxThreads to the slowest shared resource downstream, and use timeouts so a slow dependency cannot hold every thread.

Compare these tuning moves:

  • CPU-bound handlers: threads near the core count, maybe 2x.
  • I/O-bound handlers (waiting for DB or HTTP): many more threads, limited by downstream capacity.
  • Long slow requests (report downloads): move to async servlets or a separate connector so they do not starve the main pool.

Always measure with a load test before changing production values, and look at currentThreadsBusy through JMX.

Stand-in: a pool under overload

This stand-in uses a real ThreadPoolExecutor with 4 workers and a queue of 6, and submits 14 tasks that block until released. It shows the three stages: running, queued and rejected.

Java Stand-in: workers, queue and rejection

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with IO.readln() or a Scanner), it gets the next line from this box.

With 4 workers and 6 queue slots, 10 of the 14 tasks are accepted and 4 are rejected, which is the capacity limit in miniature.

Stand-in: latency as the pool saturates

The second stand-in is a deterministic simulation: requests arrive every 10 ms and each needs 100 ms of work. It computes the average response time for different pool sizes, which shows the sharp "knee" when the pool is too small.

Java Stand-in: pool size and latency

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with IO.readln() or a Scanner), it gets the next line from this box.

Below about 10 threads the queue grows without bound (latency keeps rising with the test length), right at 10 it is borderline, and above that latency settles near the 100 ms service time. Extra threads beyond what is needed give no benefit.

Timeouts and slow clients

connectionTimeout protects against clients that connect and send nothing; keepAliveTimeout frees idle connections. Behind a proxy that keeps connections open to Tomcat, set keepAliveTimeout longer than the proxy's idle timeout to avoid the proxy using a connection Tomcat just closed (a source of rare 502 errors). For slow upstream calls, set client timeouts in your own code (HTTP clients, JDBC queryTimeout) so a stuck dependency fails fast instead of consuming all threads.

HTTP/2 and compression

Add <UpgradeProtocol className="org.apache.coyote.http2.Http2Protocol"/> to a TLS connector to enable HTTP/2. Compression (compression="on") saves bandwidth for text, but if a reverse proxy already compresses, turn it off in one place to avoid double work. Do not compress images or already-compressed files.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does maxThreads limit?

    maxThreads is the size of the worker pool that runs servlet code.
  2. 2. What happens when maxConnections is reached?

    Once maxConnections is reached, further connections queue up to acceptCount, then are refused.
  3. 3. A service handles 200 requests per second at 250 ms each. About how many requests are in flight on average?

    Little's Law: 200 per second times 0.25 seconds is 50 requests in flight, before adding headroom.
  4. 4. Why is a very large maxThreads not automatically better?

    More threads add memory use and context switching, and just move the queue to a slower place.

Summary

  • Acceptor and poller threads handle connections; worker threads run your code only while a request is active.
  • Limits stack up: maxThreads, then maxConnections, then acceptCount, then refusal.
  • Size pools with Little's Law and match them to downstream capacity; measure before changing.
  • Set timeouts for clients and for your own outgoing calls.

Next: JVM memory and GC tuning for Tomcat