Skip to content
Tutorials Center
My learning

Topics

Lesson 26 of 30 ยท Real-World Projects

Project: Reverse Proxy for Several Apps

Put one Nginx or Apache entry point in front of a Node API, a Python app and a WebSocket service: path and subdomain routing, rewriting, timeouts and headers.

Real systems are rarely a single application. A product might have a Node.js API, a Python web app, a small WebSocket service for live updates and an admin tool, each listening on its own local port. Visitors should see one site, one certificate and one set of URLs. A reverse proxy is the piece that makes that possible, and this project builds the whole arrangement.

You will route by path (/api/ goes to the API) and by subdomain (admin.example.com goes to the admin tool), strip prefixes the backends should not see, upgrade connections for WebSockets, and give each service its own timeouts. The ideas come from Reverse Proxy Basics and WebSockets, Streaming and Long Requests; here you combine them.

Nginx and Apache cannot run on this site, so their configuration is shown in read-only blocks. The live examples are stand-ins: real Node.js servers on loopback plus a small hand-written proxy that follows the same routing rules. The stand-in proxy is not nginx, and it deliberately skips many features of the real one.

The plan

flowchart LR
    C["Browser"] --> P["Nginx or Apache<br/>:443, TLS"]
    P -->|"/"| W["Python app<br/>127.0.0.1:8000"]
    P -->|"/api/"| A["Node API<br/>127.0.0.1:3000"]
    P -->|"/live/"| L["WebSocket service<br/>127.0.0.1:3001"]
    P -->|"admin.example.com"| D["Admin tool<br/>127.0.0.1:9000"]

One public entry point, four private services that only listen on 127.0.0.1.

Keep every backend bound to 127.0.0.1 (or a private network) so the only way in is through the proxy. Then the proxy is the single place where TLS, rate limits and access rules live.

What the proxy must get right

Routing Pick the backend by host name and path.
Path rewrite Strip /api/ if the backend does not expect it.
Headers Pass Host, client IP and the original scheme.
Timeouts Slow reports and chat sockets need different limits.

Four decisions per service. Write them down before the config.

1 / 2

Step 1: the Nginx configuration

Start with two shared pieces at the http level: a map that decides the Connection header (for WebSockets) and one upstream per service. Naming upstreams keeps the locations short and lets you add servers later (see Load Balancing and Health Checks).

NGINX
map $http_upgrade $connection_upgrade {
    default upgrade;
    ''      close;
}

upstream web   { server 127.0.0.1:8000; keepalive 16; }
upstream api   { server 127.0.0.1:3000; keepalive 16; }
upstream live  { server 127.0.0.1:3001; }
upstream admin { server 127.0.0.1:9000; }

server {
    listen 443 ssl;
    http2 on;
    server_name example.com;
    # ssl_certificate lines omitted, see the TLS lesson

    proxy_http_version 1.1;

    # The default: the Python app
    location / {
        include snippets/proxy-headers.conf;
        proxy_set_header Connection "";
        proxy_pass http://web;
    }

    # API: strip the /api/ prefix (note the slash after the upstream name)
    location /api/ {
        include snippets/proxy-headers.conf;
        proxy_set_header Connection "";
        proxy_pass http://api/;
        proxy_read_timeout 30s;
    }

    # Long reports get more time, but only this path
    location /api/reports/ {
        include snippets/proxy-headers.conf;
        proxy_set_header Connection "";
        proxy_pass http://api/reports/;
        proxy_read_timeout 300s;
    }

    # WebSockets: keep the connection open for up to an hour of silence
    location /live/ {
        include snippets/proxy-headers.conf;
        proxy_pass http://live/;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection $connection_upgrade;
        proxy_read_timeout 3600s;
    }
}

server {
    listen 443 ssl;
    http2 on;
    server_name admin.example.com;

    location / {
        allow 10.0.0.0/8;
        deny  all;
        include snippets/proxy-headers.conf;
        proxy_pass http://admin;
    }
}

The shared file snippets/proxy-headers.conf holds the lines every route needs:

NGINX
proxy_set_header Host              $host;
proxy_set_header X-Real-IP         $remote_addr;
proxy_set_header X-Forwarded-For   $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;

Several details deserve a second look.

The slash after http://api/. When proxy_pass contains a URI part (even just /), nginx replaces the matched location prefix with it, so /api/users/7 becomes /users/7. Without it, the path is forwarded unchanged. This single character is the top cause of "my API returns 404 behind the proxy", and the stand-in later in this lesson prints all the combinations.

proxy_set_header inheritance. Directives such as proxy_set_header are inherited from the level above only if the current level sets none of its own. If you wrote the four shared lines in the server block and then added proxy_set_header Connection "" in a location, that location would silently lose the Host and X-Forwarded-* lines. The cure used above is an include file that every location pulls in, next to its own extra headers. The rule is described in Nginx Configuration Syntax.

Keepalive to the backend. keepalive 16 in the upstream plus proxy_http_version 1.1 and an empty Connection header lets nginx reuse connections to the Node API instead of opening a new one per request. The WebSocket location must not use that empty header, because it needs the real Upgrade handshake.

Three timeouts. proxy_connect_timeout, proxy_send_timeout and proxy_read_timeout default to 60 seconds. Note that proxy_read_timeout measures the gap between two reads, not the total time, so a streaming response that sends data regularly never times out. A WebSocket with no traffic does, which is why the live service either sends pings or gets a long limit.

Step 2: the same stack on Apache

Apache needs mod_proxy, mod_proxy_http and mod_proxy_wstunnel, and the Host header is not forwarded by default unless you turn on ProxyPreserveHost:

Terminal
sudo a2enmod proxy proxy_http proxy_wstunnel headers
APACHE
<VirtualHost *:443>
    ServerName example.com
    # SSLEngine and certificate lines omitted

    ProxyPreserveHost On
    RequestHeader set X-Forwarded-Proto "https"

    # Order matters: the first matching ProxyPass wins, so specific paths go first
    ProxyPass        "/api/reports/" "http://127.0.0.1:3000/reports/" timeout=300
    ProxyPassReverse "/api/reports/" "http://127.0.0.1:3000/reports/"

    ProxyPass        "/api/"  "http://127.0.0.1:3000/" timeout=30
    ProxyPassReverse "/api/"  "http://127.0.0.1:3000/"

    ProxyPass        "/live/" "ws://127.0.0.1:3001/" timeout=3600
    ProxyPassReverse "/live/" "ws://127.0.0.1:3001/"

    ProxyPass        "/" "http://127.0.0.1:8000/"
    ProxyPassReverse "/" "http://127.0.0.1:8000/"
</VirtualHost>

<VirtualHost *:443>
    ServerName admin.example.com
    ProxyPreserveHost On
    ProxyPass        "/" "http://127.0.0.1:9000/"
    ProxyPassReverse "/" "http://127.0.0.1:9000/"
    <Location "/">
        Require ip 10.0.0.0/8
    </Location>
</VirtualHost>

The differences from nginx are instructive. Apache adds X-Forwarded-For for you with mod_proxy, but not the scheme, so you set X-Forwarded-Proto yourself. Routing order is file order for ProxyPass, so the reports line must come before /api/; nginx instead picks the longest matching prefix regardless of order. ProxyPassReverse rewrites Location headers in redirects so a backend that answers Location: http://127.0.0.1:3000/users/7 does not leak its private address to the browser. Nginx has proxy_redirect for the same job, and its default is usually right when Host is passed along.

Step 3: see the routing work

The stand-in below starts three real HTTP backends (an API, a web app and a deliberately slow "reports" service) on random loopback ports, then runs a tiny proxy in front of them. Its routing table is the same idea as the configuration above: match host and prefix, strip the prefix, add forwarding headers, and apply a per-route timeout. Each backend replies with what it received, so you can see exactly what the proxy changed.

Node.js Stand-in: a multi-backend proxy

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with node:readline or process.stdin), it gets the next line from this box.

Read the output closely. The API saw /users/7, not /api/users/7: the prefix was stripped. Every backend saw the original Host header and a client address in X-Forwarded-For. The reports route returned a 504 because the stand-in gives it a 150 ms limit and the service takes 300 ms. In production you would not set a limit that low, of course, but the point stands: the timeout belongs to the route, so a slow report can be allowed 300 seconds while the rest of the API stays at 30. A 502 would mean the backend refused or dropped the connection (not running, wrong port), while a 504 means it was reachable but too slow. Try raising timeoutMs for the reports route to 500 and run again.

Step 4: get the slash right

Path rewriting is where most proxy bugs hide, so test it before you deploy. The function below reproduces how nginx builds the upstream path for a plain prefix location and a proxy_pass without variables. It also shows the tiny map that sets the WebSocket Connection header.

JS Stand-in: proxy_pass and the trailing slash

Press Run (or Ctrl + Enter) to see the console output here.

No output.

The rows with odd results are the lesson. location /api with proxy_pass http://...:3000/ turns /api/users/7 into //users/7 (a double slash) and /apiary into /ary, because the prefix match is textual. The last row loses the slash between v2 and users. The rule that avoids all of these: end the location and the proxy_pass URI with a slash together, or give neither one.

Step 5: verify the finished stack

On the real server, test each route from the outside and from the backend's point of view:

Terminal
sudo nginx -t && sudo systemctl reload nginx

curl -i https://example.com/api/health          # API, prefix stripped
curl -i -H "Host: admin.example.com" https://127.0.0.1/ -k   # from outside the allow list: 403
curl -i -N \
  -H "Connection: Upgrade" -H "Upgrade: websocket" \
  -H "Sec-WebSocket-Version: 13" -H "Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==" \
  https://example.com/live/socket               # expect 101 Switching Protocols

Add a log line that records the upstream address and timings so slow routes are visible ($upstream_addr, $upstream_response_time; see Logs and Troubleshooting). If a backend sees the wrong client IP or builds http:// links on an HTTPS site, it is not reading X-Forwarded-For and X-Forwarded-Proto; frameworks have a "trusted proxies" setting for exactly this, and it should list only your proxy.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. In nginx, location /api/ { proxy_pass http://api/; } receives /api/users/7. What path does the backend see?

    proxy_pass has a URI part (the trailing slash), so nginx replaces the matched prefix /api/ with / and forwards /users/7.
  2. 2. Which status code means the proxy reached the backend but waited longer than the configured read timeout?

    504 is a timeout. 502 usually means the backend refused the connection, closed it early or sent an invalid reply.
  3. 3. Why does the WebSocket location use proxy_set_header Connection $connection_upgrade instead of the empty Connection header used for keepalive?

    Upgrade and Connection are hop-by-hop headers, so nginx does not pass them unless you set them. The map sends 'upgrade' when the client asked for it and 'close' otherwise.
  4. 4. In Apache, why must the ProxyPass line for /api/reports/ come before the one for /api/?

    Unlike nginx's longest-prefix choice, ProxyPass is evaluated in the order written, so the specific path must be listed first.

Summary

  • One proxy with TLS in front of private backends gives visitors a single site; bind the backends to 127.0.0.1.
  • Route by server_name (subdomains) and by location or ProxyPass (paths); nginx picks the longest prefix, Apache the first match in file order.
  • A trailing slash on both the location and the proxy_pass URI strips the prefix; mismatched slashes cause double slashes and 404s.
  • Pass Host, the client IP and X-Forwarded-Proto, and keep shared headers in an include so inheritance does not drop them.
  • WebSockets need Upgrade and Connection headers plus a long read timeout; give slow routes their own timeouts.
  • 502 means a broken or refused backend connection, 504 means a slow backend.

Next: Project: A Secure Load-Balanced Production Setup.