Skip to content
Tutorials Center
My learning

Topics

Lesson 27 of 30 ยท Real-World Projects

Project: A Secure Load-Balanced Production Setup

Assemble a production edge: TLS termination, an upstream pool with passive health checks, rate limiting, caching, security headers, useful logs and a safe reload routine.

This is the capstone project of the course. You will assemble a production-grade edge server out of the pieces from earlier lessons: TLS terminated at the proxy, a pool of application servers behind it, protection against abusive clients, a short-lived cache for repeatable responses, security headers, logs that help during an incident, and a routine for changing the configuration without taking the site down.

The design goal is not cleverness. It is a configuration that you can read top to bottom, that fails in predictable ways, and that a colleague can change at three in the morning without guessing. Each section below adds one layer and explains what that layer protects you from. The layers are covered in depth in HTTPS, TLS and Let's Encrypt, Load Balancing and Health Checks, Access Control and Rate Limiting and Hardening and Security Headers.

Nginx and Apache cannot run on this site, so the real configuration is read-only. The live examples are JavaScript stand-ins: a small simulation that applies rate limiting, caching and balancing in the same order as the real edge, and an auditor that checks a config text against a checklist. They show the logic, not the real server.

The architecture

flowchart LR
    U["Clients"] -->|"HTTPS :443"| E["Edge: TLS, rate limit,<br/>cache, headers"]
    E --> A1["app1 :8080"]
    E --> A2["app2 :8080"]
    E -.->|"backup only"| A3["app3 :8080"]

Clients speak HTTPS to the edge; the edge speaks plain HTTP over the private network to the pool.

The edge, layer by layer

TLS Certificates, modern protocols, HSTS.
Rate limit Slow down abusive clients early.
Cache Answer repeat requests without the app.
Pool Spread traffic, skip dead servers.

Order matters. Cheap checks first, expensive work last.

1 / 2

Layer 1: TLS termination and the redirect

The edge holds the certificate and decrypts traffic. Plain HTTP exists only to redirect, and to answer the certificate authority's challenge if you use HTTP validation (certbot's nginx and Apache plugins add that for you).

NGINX
server {
    listen 80;
    server_name example.com www.example.com;
    return 301 https://example.com$request_uri;
}

server {
    listen 443 ssl;
    http2 on;
    server_name example.com;

    ssl_certificate     /etc/letsencrypt/live/example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;
    ssl_protocols       TLSv1.2 TLSv1.3;
    ssl_session_cache   shared:SSL:10m;
    ssl_session_timeout 1d;
    ssl_session_tickets off;

    server_tokens off;
    # ... the layers below go here
}

fullchain.pem contains your certificate and the intermediates, so clients can build the chain. ssl_protocols drops the obsolete TLS 1.0 and 1.1. The shared session cache lets returning clients skip the expensive part of the handshake. If you also serve www, give it its own server block with a redirect to the canonical host, and its own certificate coverage, as described in Redirects and Rewrites.

On Apache the same layer uses mod_ssl:

APACHE
<VirtualHost *:80>
    ServerName example.com
    Redirect permanent / https://example.com/
</VirtualHost>

<VirtualHost *:443>
    ServerName example.com
    Protocols h2 http/1.1
    SSLEngine on
    SSLCertificateFile    /etc/letsencrypt/live/example.com/fullchain.pem
    SSLCertificateKeyFile /etc/letsencrypt/live/example.com/privkey.pem
    SSLProtocol           -all +TLSv1.2 +TLSv1.3
    ServerSignature Off
</VirtualHost>

(ServerTokens Prod is a server-wide setting, so it goes in the main configuration, not in the vhost.)

Layer 2: the upstream pool

Define the pool once, and let nginx notice failures on its own:

NGINX
upstream app_pool {
    least_conn;
    server 10.0.0.11:8080 max_fails=2 fail_timeout=10s;
    server 10.0.0.12:8080 max_fails=2 fail_timeout=10s;
    server 10.0.0.13:8080 backup;
    keepalive 32;
}

This is passive health checking: nothing probes the servers. When real requests to a server fail max_fails times within fail_timeout, nginx stops sending it traffic for fail_timeout, then tries again. Meanwhile proxy_next_upstream (defaults to error timeout) retries the same request on the next server, so one crashed machine usually costs the client a small delay and not an error. Retries are only safe for requests that can be repeated, which is why nginx does not retry POST requests after the request was sent. The backup server only receives traffic when all the others are unavailable. Probing servers actively (a /health check every few seconds) needs a third-party module or the commercial NGINX Plus in nginx's case, and mod_proxy_hcheck in Apache's.

Apache's pool is a balancer:

APACHE
<Proxy "balancer://app_pool">
    BalancerMember "http://10.0.0.11:8080" retry=10
    BalancerMember "http://10.0.0.12:8080" retry=10
    BalancerMember "http://10.0.0.13:8080" status=+H
    ProxySet lbmethod=bybusyness
</Proxy>
ProxyPreserveHost On
RequestHeader set X-Forwarded-Proto "https"
ProxyPass        "/" "balancer://app_pool/"
ProxyPassReverse "/" "balancer://app_pool/"

status=+H marks a hot standby and retry=10 is the number of seconds before a failed member is tried again.

Layer 3: rate limiting and a cache

Rate limiting protects the pool from clients that send too much, whether a buggy script or an attack. In nginx the zone is declared once at the http level and applied where needed:

NGINX
limit_req_zone  $binary_remote_addr zone=perip:10m rate=10r/s;
limit_req_status 429;

location / {
    limit_req zone=perip burst=20 nodelay;
    proxy_pass http://app_pool;
}

rate=10r/s is the long-term average. burst=20 lets a client momentarily exceed it by 20 requests, which is what a page that loads many assets does. With nodelay the burst is served immediately and then the client is rejected until the bucket drains. Apache has no built-in per-client request-rate limiter in its standard modules (mod_ratelimit limits bandwidth, not request counts), so teams either keep the nginx edge or add a module such as mod_qos, or limit at a CDN or WAF.

A short cache turns repeated reads into memory-speed answers and keeps working when the app hiccups:

NGINX
proxy_cache_path /var/cache/nginx levels=1:2 keys_zone=edge:10m
                 max_size=1g inactive=60m use_temp_path=off;

location /api/public/ {
    proxy_cache edge;
    proxy_cache_valid 200 10s;
    proxy_cache_lock on;
    proxy_cache_use_stale error timeout updating http_502 http_503 http_504;
    add_header X-Cache-Status $upstream_cache_status always;
    proxy_pass http://app_pool;
}

Cache only responses that are the same for everyone. Anything with a session cookie or a personal cart must bypass it (nginx will not cache responses carrying Set-Cookie unless told to). proxy_cache_lock on makes concurrent misses for the same URL wait for one backend request, and use_stale serves a slightly old copy while the backend is failing. Apache's counterpart is mod_cache with mod_cache_disk (CacheEnable disk "/api/public/").

Layer 4: headers and logs

Headers are cheap, so set them for the whole site. In nginx remember the inheritance trap from Nginx Configuration Syntax: add_header in a location replaces the ones from the server level. Keep them in one include and use always so they are also sent on error responses:

NGINX
# snippets/security-headers.conf
add_header Strict-Transport-Security "max-age=31536000" always;
add_header X-Content-Type-Options    "nosniff" always;
add_header X-Frame-Options           "SAMEORIGIN" always;
add_header Referrer-Policy           "strict-origin-when-cross-origin" always;

Raise the HSTS lifetime only once you are certain every subdomain works over HTTPS, because browsers will refuse plain HTTP for that whole time. Add a Content-Security-Policy only after testing it against your pages, since a wrong one breaks the site (see Hardening and Security Headers).

For logs, add the fields you will want during an incident: how long the edge and the upstream took, which server answered, and whether the cache helped.

NGINX
log_format edge '$remote_addr [$time_local] "$request" $status $body_bytes_sent '
                'rt=$request_time urt=$upstream_response_time '
                'ua=$upstream_addr cache=$upstream_cache_status';
access_log /var/log/nginx/edge.access.log edge;

With this format, a slow page shows rt=2.3 and ua=10.0.0.12:8080, which tells you at once whether the edge or one backend is the problem. Logs and Troubleshooting shows how to analyze these lines.

Watching the layers work

The simulation below applies the layers in the same order the real edge does: rate limit first (cheapest), then the cache, then the balancer. One client sends a burst of ten requests to an uncacheable path, the pool contains a crashed server (app2), and a handful of ordinary clients follow. Read the log, then change RATE, BURST or down: true and run it again.

JS Stand-in: traffic through the edge

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Notice four things in the output. The burst gets six answers and then four 429 responses: that is burst=5 plus the first request. app2 never serves anything, yet no client saw an error: each request that landed on it was retried on the next server, and after two failures it was parked for 10 virtual seconds. The cacheable paths show cache HIT after the first miss, while /cart always goes to a backend. Finally the totals add up: every request ended in exactly one outcome.

Layer 5: the reload checklist

A configuration is only as safe as the routine used to apply it. Use the same four steps every time:

Terminal
sudo nginx -t                         # 1. syntax and file checks (Apache: apachectl configtest)
sudo nginx -T | less                  # 2. dump the effective config when something surprises you
sudo systemctl reload nginx           # 3. graceful: new workers start, old ones finish their requests
curl -sI https://example.com/ | head  # 4. smoke test: status, HSTS header, cache status

A failed nginx -t must stop the sequence, so chain with && in scripts. After the reload, watch the error log for a minute and request one URL per layer: the redirect, a cached path, an uncached path and a path you know is rate limited. Reloads, Zero-Downtime Deploys and Containers explains what happens to in-flight requests during step 3.

The second stand-in turns the checklist into code. It scans a configuration text and reports which layers are missing. It cannot replace nginx -t, but it is a quick review aid, and the same idea can run in a CI pipeline against your real files.

JS Stand-in: audit a config against the checklist

Press Run (or Ctrl + Enter) to see the console output here.

No output.

The sample scores 7 of 11, and the misses are real bugs: the port-80 block proxies instead of redirecting, the limit_req line refers to a zone that was never declared (nginx -t would refuse that config), and there is no cache or timing log. Paste the config from this lesson into the string and the score rises. Adding a check for every incident your team has had is a cheap way to turn painful lessons into permanent safeguards.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. With max_fails=2 and fail_timeout=10s, what does nginx do after a server fails twice within 10 seconds?

    This is passive health checking. The server is considered unavailable for fail_timeout and is then given another chance with live traffic.
  2. 2. In the rate limit limit_req zone=perip burst=20 nodelay, what is the role of burst?

    rate is the long-term speed; burst is the queue size of extra requests allowed in a short spike. With nodelay those extra requests are served immediately.
  3. 3. Why should security headers use 'always' and live in an include file in nginx?

    By default add_header applies only to successful and redirect responses, and a location with its own add_header drops those from the level above. An include with 'always' fixes both problems.
  4. 4. What is the safest order for applying a config change?

    A test catches syntax and file errors before anything changes, a reload keeps serving existing connections, and a smoke test confirms that the live behavior is what you intended.

Summary

  • Terminate TLS at the edge, redirect HTTP to HTTPS, allow only TLS 1.2 and 1.3, and add HSTS when you are sure.
  • Describe the pool with upstream (or an Apache balancer); max_fails and fail_timeout give passive health checks, and backup servers cover total failure.
  • Rate limit with a rate plus a burst; cache only responses that are identical for every user.
  • Keep headers in an include with always, and log timings, upstream address and cache status.
  • Apply every change with test, graceful reload and smoke test.
  • Encode your checklist as code so it is checked every time, not only when someone remembers.

Next: Practice: Configuration Exercises with Solutions.