Skip to content
Tutorials Center
My learning

Topics

Lesson 10 of 65 ยท Building Blocks of the Web Stack

Web Servers and Application Servers

Learn the difference between web and application servers, how thread pools and event loops handle concurrency, and size a worker pool with Python.

When a browser asks for a page, the request lands on a server. But "server" hides two quite different jobs. One is fast, generic and mostly about moving bytes: serving files, terminating TLS, forwarding requests. The other runs your business logic: checking a password, pricing a cart, querying a database. The first is the web server, the second the application server. Understanding how each handles many users at once is the foundation for scaling the middle of any web system.

Web server

A web server accepts HTTP connections and answers requests. Its core skills are:

  • Serving static content (HTML, CSS, JavaScript, images, downloads) directly from disk or memory, very efficiently.
  • Handling thousands of concurrent connections, including slow clients.
  • TLS termination, compression, HTTP/2, access logs.
  • Acting as a reverse proxy, passing dynamic requests to application servers (see proxies).
  • URL rewriting, redirects, basic access control and rate limiting.

Examples: Nginx, Apache HTTP Server, Caddy, IIS. A static site needs nothing more than a web server (or a CDN in front of one).

Application server

An application server runs the program that produces dynamic responses. It hosts your code, manages worker processes or threads, connects to databases and caches, and returns the generated page or JSON. Examples depend on the language: Gunicorn or uvicorn for Python, PHP-FPM for PHP, Node.js with Express, Tomcat or Spring Boot for Java, Puma for Ruby, Kestrel for .NET.

Many frameworks blur the line (Node and Go programs can serve HTTP directly), yet in production it is still common to put a web server in front for TLS, static files, buffering and protection.

flowchart LR
  B["Browser"] --> W["Web server<br/>Nginx"]
  W -->|"static files"| F[("Disk / CDN")]
  W -->|"dynamic requests"| A1["App server 1"]
  W --> A2["App server 2"]
  A1 --> DB[("Database")]
  A2 --> DB

Figure: the web server handles the front line and static assets; app servers run business logic.

Web server vs app server

Web server Static files, TLS, proxying, many connections.
App server Runs your code and talks to the database.
Together Web server in front, app servers behind.

Small apps may run both roles in one process.

1 / 2

The multi-tier idea

Systems are commonly organized in tiers, each with one responsibility and its own scaling:

  1. Presentation tier: browsers, mobile apps, and the web servers that deliver them.
  2. Application (logic) tier: app servers running business rules.
  3. Data tier: databases, caches and file storage.

Splitting tiers lets you scale each independently (many cheap web and app servers, a carefully protected data tier), restrict network access (only the app tier may talk to the database), and change one tier without rewriting the others.

How a server handles concurrency

A server that handled one request at a time would be useless, because most requests spend most of their time waiting (on a database, on another service, on disk). There are a few standard models.

Process-per-request or process pool: each request goes to a separate worker process (classic PHP-FPM, Gunicorn sync workers). Isolated and simple; memory heavy, so you cap the number of workers.

Thread pool: one process, a fixed set of threads, each handling one request at a time (Tomcat, many Java and .NET servers). Cheaper than processes; threads blocked on I/O still occupy a slot.

Event loop (asynchronous I/O): one thread juggles thousands of connections, switching to another request whenever one is waiting on I/O (Node.js, Nginx, async Python). Very efficient for I/O-bound work, but one long CPU-bound task blocks everyone.

Model Strength Weakness
Process pool Isolation, simple High memory per worker
Thread pool Good balance Blocking uses up threads
Event loop Huge concurrency for I/O CPU-heavy work stalls it

Sizing a worker pool

By Little's law, the number of requests in flight equals arrival rate times time-in-system. If requests take 80 ms and you want to serve 500 per second, you need about 500 x 0.08 = 40 workers busy at once, and more for headroom. Too few workers and requests queue; too many and you waste memory or thrash.

Python How many workers do I need?

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

With 36 workers capacity is below demand, so waits grow without bound. At 40 it is almost exactly at the limit and waits are noticeable; with 55 to 70 the queue nearly disappears. The sweet spot has about 30% headroom, which matches the advice to avoid running near full utilization.

Blocking versus non-blocking

When a request waits 200 ms for a database, a blocking worker does nothing for that time. An event-loop server uses the wait to serve others. The simulation compares how many requests per second one worker can complete in each style when the work is mostly waiting.

Python Blocking worker vs event loop

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

For I/O-bound work the single event loop outperforms eight blocking workers by a wide margin, but when the work is CPU-bound a single thread is the limit and you need more processes. That is why Node and async Python are usually run as several processes, one per core.

Practical guidance

  • Put a reverse proxy or web server in front of application servers for TLS, buffering and static files.
  • Serve static assets from a CDN, not from your application servers.
  • Keep app servers stateless so you can add or remove them freely (stateless systems).
  • Set timeouts everywhere: a hung dependency must not hold workers forever.
  • Configure graceful shutdown so deployments finish in-flight requests.
  • Tune pool sizes with load tests, and watch queue length as well as CPU.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Which task fits a web server best?

    Static files and connection handling are the web server's strength; business logic belongs to the application tier.
  2. 2. Why put a web server or reverse proxy in front of app servers?

    App workers are valuable; do not let them wait on slow connections or serve plain files.
  3. 3. An app does mostly I/O waiting. Which model gives the most concurrency per thread?

    An event loop uses wait time to serve other requests.
  4. 4. Requests take 100 ms and arrive at 300 per second. About how many workers are busy on average?

    300 x 0.1 = 30 requests in flight, so you need more than 30 workers for headroom.

Summary

  • Web servers handle connections, TLS and static files; application servers run business logic.
  • Separate tiers (presentation, application, data) so each scales and is secured independently.
  • Concurrency models: process pools, thread pools and event loops, each with trade-offs.
  • Size worker pools using Little's law plus headroom; set timeouts and shut down gracefully.
  • Keep application servers stateless so they scale out behind a load balancer.

Next: Stateless and Stateful Systems