Skip to content
Tutorials Center
My learning

Topics

Lesson 22 of 30 ยท Operations and Performance

Performance Tuning

Tune nginx and Apache for speed: workers and connections, keep-alive, open file cache, buffers, Apache event MPM limits, proxy and FastCGI caching.

Nginx and Apache are fast with default settings for a small site. Performance tuning matters when traffic grows, when each request is expensive (a PHP page that hits a database), or when memory is tight. The usual lesson is humbling: the biggest speedups rarely come from clever kernel flags. They come from not doing the work at all (caching), from not wasting connections (keep-alive, sensible limits), and from knowing your numbers before changing anything.

This lesson works from the cheap and effective to the fine print: workers and connection limits, keep-alive, file and buffer settings, Apache's event MPM limits, response caching with proxy_cache and the FastCGI cache, and finally how to load test so you can prove a change helped.

Real nginx and Apache configuration appears in read-only blocks, because the site cannot run a web server. The interactive examples are stand-ins: a capacity calculator and a small cache simulation in Node that show why the settings behave the way they do. Their numbers are illustrations, not benchmarks of nginx or Apache.

Where speed comes from

Cache The fastest request never reaches the backend.
Reuse Keep-alive avoids repeated connection setup.
Size limits Match workers to cores, RAM and traffic.
Measure Load test before and after every change.

Change one thing at a time, and keep what the numbers prove.

1 / 2

Workers and connections in nginx

nginx runs a master process and a few worker processes, each one an event loop that handles thousands of connections without a thread per connection (the architecture is compared in Nginx vs Apache). Two settings size it:

NGINX
worker_processes auto;            # one worker per CPU core

events {
    worker_connections 4096;      # connections per worker
}

auto is almost always right: more workers than cores just adds context switching. worker_connections is the cap on simultaneous connections per worker, and the built-in default is low (512; many packages set 768 or 1024). The theoretical ceiling is worker_processes x worker_connections, but remember that when nginx proxies, every client uses two connections (one from the client, one to the backend), so halve your estimate.

Each connection also needs a file descriptor. If the error log says Too many open files (or worker_connections are not enough), raise the limit with worker_rlimit_nofile 65535; in the main context and make sure the service's own limit (LimitNOFILE in the systemd unit) allows it.

Keep-alive: stop paying for new connections

Opening a TCP (and TLS) connection costs round trips. Keep-alive lets a browser send many requests over one connection:

NGINX
keepalive_timeout  15s;           # how long an idle connection stays open
keepalive_requests 1000;          # requests allowed per connection

Longer is not automatically better: an idle connection occupies a slot (cheap in nginx, expensive in a process-per-connection Apache). A timeout of 5 to 30 seconds is a sensible range. The same idea applies on the backend side, where it often matters more, because connecting to your own application for every request is pure overhead:

NGINX
upstream app {
    server 127.0.0.1:3000;
    keepalive 32;                         # idle connections kept per worker
}

location / {
    proxy_pass http://app;
    proxy_http_version 1.1;               # keep-alive needs HTTP/1.1 ...
    proxy_set_header Connection "";       # ... and no "Connection: close"
}

Apache: KeepAlive On, MaxKeepAliveRequests 100, KeepAliveTimeout 5 are the matching knobs. Apache's default of 5 seconds is deliberate, for the reason above.

File handling and buffers

For static files, three switches let the kernel do the work:

NGINX
sendfile    on;                   # copy file to socket inside the kernel
tcp_nopush  on;                   # with sendfile: send full packets
tcp_nodelay on;                   # do not delay small packets on keep-alive connections

Files must be found, opened and stated on every request. When a site serves many small files, cache those lookups:

NGINX
open_file_cache          max=2000 inactive=20s;
open_file_cache_valid    30s;
open_file_cache_min_uses 2;
open_file_cache_errors   on;

The trade-off: for up to open_file_cache_valid seconds, nginx may not notice that a file was replaced. If you deploy by swapping a directory (see Reloads, Zero-Downtime Deploys and Containers), keep that value short.

Buffers hold requests and responses in memory. If something does not fit, nginx writes it to a temporary file on disk, which is slow, and you will see a client request body is buffered to a temporary file or an upstream response is buffered to a temporary file in the error log. Settings to know are client_body_buffer_size, proxy_buffer_size and proxy_buffers 8 16k. Raise them only when logs show spills, because buffers are per connection. Compression is covered in HTTP/2, HTTP/3, gzip and Brotli.

Apache: the event MPM limits

Apache's speed is governed by its MPM (see Apache httpd Configuration). With the event MPM, a handful of numbers decide capacity:

APACHE
<IfModule mpm_event_module>
    ServerLimit              16
    ThreadsPerChild          25
    MaxRequestWorkers       400     # = ServerLimit x ThreadsPerChild
    MinSpareThreads          75
    MaxSpareThreads         250
    MaxConnectionsPerChild 10000    # recycle processes to contain slow memory leaks; 0 = never
</IfModule>

MaxRequestWorkers is the number of requests Apache will process at the same moment. When it is reached, further connections wait in a queue (and the error log notes that the limit was reached). Setting it higher than your RAM allows is the classic mistake: the machine starts swapping and everything gets slower. A safe way to pick it, especially with prefork, is:

MaxRequestWorkers โ‰ˆ memory you can give Apache / average memory of one worker process

Measure the average with ps -ylC apache2 --sort:rss while the site is under typical load. With the event MPM and PHP-FPM (see PHP, FastCGI and Application Servers), Apache holds no PHP interpreter in its workers, so the processes are small and the limit is mainly a concurrency choice.

Why do these architectures need such different resources? The sizing calculator below uses a rough worst-case model (every request opens a connection that then idles for the keep-alive time). It shows that the number of busy requests is small while the number of open connections is large, which is exactly the gap event-driven designs exploit.

JS Stand-in: connection sizing calculator

Press Run (or Ctrl + Enter) to see the console output here.

No output.

The first line of each block is Little's law at work: 50 requests per second at 80 ms each means only 4 requests are being processed at any instant, even though up to 254 connections may be open. Shortening the keep-alive (third case) cuts open connections from 2,048 to 449 at the same traffic. The memory figures are invented round numbers for the demonstration. Measure your own.

Caching responses: the biggest lever

If a page is the same for thousands of visitors, generating it once and replaying it is hundreds of times cheaper than running PHP and a database each time. nginx can store backend responses on disk and serve them directly:

NGINX
http {
    proxy_cache_path /var/cache/nginx/app levels=1:2 keys_zone=appcache:10m
                     max_size=1g inactive=60m use_temp_path=off;

    server {
        location / {
            proxy_pass http://app;
            proxy_cache appcache;
            proxy_cache_valid 200 301 10m;           # keep good answers for 10 minutes
            proxy_cache_valid 404 1m;
            proxy_cache_lock on;                      # one request fetches, the others wait
            proxy_cache_use_stale error timeout updating http_500 http_502 http_503 http_504;
            proxy_cache_background_update on;         # refresh stale entries in the background
            add_header X-Cache-Status $upstream_cache_status always;
        }
    }
}

The X-Cache-Status header (HIT, MISS, EXPIRED, STALE, BYPASS) is the way to see what happened to a given request with curl -I. For PHP-FPM sites the same machinery exists under different names: fastcgi_cache_path, fastcgi_cache, fastcgi_cache_key and fastcgi_cache_valid.

flowchart TD
    A["Request"] --> B{"In cache and fresh?"}
    B -- "yes" --> C["Send cached copy (HIT)"]
    B -- "no" --> D{"Someone already fetching?"}
    D -- "yes (lock)" --> E["Wait, then send result"]
    D -- "no" --> F["Ask backend (MISS)"]
    F --> G["Store copy"]
    G --> H["Send response"]

A cached request skips the backend; with the lock on, only one of many simultaneous requests for a cold page reaches it.

Two points deserve emphasis. First, never cache personalized pages for everyone. By default nginx honors Cache-Control: private and no-cache and does not store responses that set cookies, but it is easy to override that by accident. Use proxy_cache_bypass and proxy_no_cache with a condition (such as a session cookie) so logged-in users always get fresh, private pages. Second, proxy_cache_lock prevents a cache stampede, where a popular page expires and hundreds of simultaneous requests all rush to the backend.

The simulation below implements a tiny cache and shows both ideas: hits are fast, a hit ratio near 99% after warm-up, and the stampede with and without a lock.

Node.js Stand-in: cache hit versus miss

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

Apache has an equivalent in mod_cache with mod_cache_disk (CacheEnable disk "/", CacheRoot), and mod_cache can lock too (CacheLock On). Many sites find it simpler to let nginx or a CDN do the caching in front of Apache (see Nginx in Front of Apache). Static files get a cheaper form of caching through browser cache headers, covered in Static Files, Caching and MIME Types.

One Apache-specific win: .htaccess files make Apache look for and parse a file in every directory on the path to each request. Setting AllowOverride None and moving rules into the main config removes that cost (see .htaccess, Directory Rules and AllowOverride).

Load testing basics

You cannot tune what you cannot measure. A load-testing tool sends many requests and reports requests per second and latency percentiles:

Terminal
ab -n 2000 -c 50 https://staging.example.com/          # ApacheBench: 2000 requests, 50 at a time
wrk -t4 -c100 -d30s https://staging.example.com/       # wrk: 4 threads, 100 connections, 30 seconds

Rules that keep tests honest:

  • Test a staging copy, never production by surprise. Tell whoever owns the hosting if it is shared.
  • Run the tool from a different machine, so it does not steal the server's CPU.
  • Look at percentiles (p95, p99), not only the average, and at the error rate. A fast average with 5% errors is a failure.
  • Change one setting per run, repeat each run, and watch the server too (top, ss -s, the error log).
  • Test the real path: a cached static page proves little about a page that renders from a database.
  • Warm the cache first if you want steady-state numbers, and test a cold cache separately.

Before tuning, check what is actually slow. The rt= and urt= fields from Logs and Troubleshooting tell you whether time goes into the backend or into the web server and the client's network. Tuning nginx will not fix a 2-second database query.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What does worker_processes auto; do in nginx?

    Each nginx worker is an event loop that handles many connections, so one per core is the usual sweet spot.
  2. 2. Why does nginx need proxy_http_version 1.1; and proxy_set_header Connection ""; for upstream keep-alive?

    Without them, each proxied request opens and closes its own connection to the backend, which wastes time and sockets.
  3. 3. What problem does proxy_cache_lock on; solve?

    When a popular entry expires, many simultaneous requests would hit the backend. The lock lets one fetch and shares the result.
  4. 4. How should you choose MaxRequestWorkers for Apache?

    Too high a value causes swapping, which is slower than queueing. Pick it from measured memory use and confirm it with a load test.
  5. 5. Which is a mistake when load testing?

    If several settings change at once you cannot tell which one made the difference. Change one thing per run.

Summary

  • Use worker_processes auto;, a realistic worker_connections, and raise file-descriptor limits when the log says Too many open files.
  • Keep-alive saves connection setup, on the client side and on the upstream side (keepalive, proxy_http_version 1.1, empty Connection header).
  • sendfile, open_file_cache and sensible buffers speed up static files; buffer spills to disk show up in the error log.
  • For Apache's event MPM, MaxRequestWorkers = ServerLimit x ThreadsPerChild; size it from memory and measure.
  • proxy_cache and fastcgi_cache skip the backend entirely; use proxy_cache_lock, use_stale, and bypass rules for logged-in users. Check results with X-Cache-Status.
  • Load test on staging, from another machine, one change at a time, and compare percentiles and errors.

Next: Reloads, Zero-Downtime Deploys and Containers.