Skip to content
Tutorials Center
My learning

Topics

Lesson 1 of 38 ยท Getting Started

What Is Queueing Theory?

Meet queueing theory: the math of waiting lines. See why queues form, why waits explode near full capacity, and simulate your first queue.

Every day you wait in lines: at a checkout, at a traffic light, on hold with a help desk. Computers wait too. Web requests wait for a free server, and data packets wait for a network link. Queueing theory is the branch of applied probability that describes these waiting lines with math, so we can predict how long things wait and decide how many servers we need.

This track starts from zero. You need only basic probability (averages and the idea of a random variable) and a little Python. We build the ideas step by step, and every formula is also checked by running code, so you never have to take a result on trust.

Why do queues form at all?

Suppose a barista serves exactly one customer every 2 minutes, and exactly one customer arrives every 2 minutes. Nobody ever waits. Now make the arrivals random but keep the same average: one customer per 2 minutes. Sometimes three customers arrive within a minute, and the barista falls behind. Sometimes nobody comes for five minutes, and the barista sits idle. The idle time is lost forever, but the backlog from a burst is not. That asymmetry is the heart of queueing theory:

Queues are caused by randomness, not only by lack of capacity. Even when a server is faster than the average demand, random bursts create waiting.

A second surprise follows: waiting time does not grow in a straight line as the server gets busier. It grows slowly at first, then explodes as the server approaches 100% busy. We will prove this in the M/M/1 chapter. For now, you will see it in a simulation.

The vocabulary in one picture

flowchart LR
  A["Arrivals<br/>(customers)"] --> Q["Waiting line<br/>(queue)"]
  Q --> S["Server(s)"]
  S --> D["Departures"]

A basic queueing system: customers arrive, wait if all servers are busy, get served, and leave.

The three parts are the arrival process (how customers come), the queue (where they wait and in what order), and the service mechanism (how many servers and how long service takes). The next lesson looks at each part more closely.

Your first simulation

Let us build the simplest possible queue: one server, random arrivals, random service. We track each customer: when they arrive, when service starts, and how long they wait. A customer starts service at the later of two moments: their own arrival, or the moment the previous customer finishes.

Python Simulate one server

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The server can handle 1 customer per minute on average, and 0.8 arrive per minute, so it is busy about 80% of the time. Yet many customers still wait, and the longest wait is far above the average. Later we will show that the theoretical average wait here is exactly 4 minutes, and your simulation should land close to it.

The explosion near full capacity

Now we keep the service rate at 1 per minute and raise the arrival rate. The average wait of a customer in a one-server queue with random arrivals and service turns out to be ฯ / (ฮผ - ฮป), where ฯ = ฮป/ฮผ is the fraction of time the server is busy. You will derive it later; here we only tabulate it.

Python How waiting grows with busyness

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Going from 50% to 80% busy multiplies the wait by four. Going from 90% to 99% multiplies it by eleven. This is why a call center aiming for 99% utilization has terrible service, and why engineers aim for spare capacity.

What queueing theory gives you

  • Prediction: average wait, queue length, and the fraction of customers who wait more than some limit.
  • Design: how many servers, how fast, how much buffer space.
  • Insight: which changes matter most (a faster server, a shared line, shorter variability).

Check your understanding

Quick check ยท 3 questions

of 3 answered You scored / 3 ยท perfect!

  1. 1. Why can a queue form even when the server is faster than the average arrival rate?

    Idle time during quiet periods cannot be saved, but a backlog from a burst remains. Randomness alone creates waiting.
  2. 2. What happens to the average wait as a single server approaches 100% busy?

    The wait formula rho / (mu - lambda) has the small difference mu - lambda in the denominator, so it blows up as utilization nears 1.
  3. 3. A customer arrives at time 10 and the server finishes the previous customer at time 13. When does this customer start service?

    Service starts at the later of the arrival time and the time the server becomes free: max(10, 13) = 13.

Summary

  • Queueing theory uses probability to predict waiting in lines of all kinds.
  • Randomness in arrivals and service creates queues even with spare capacity on average.
  • A system has an arrival process, a queue and a service mechanism.
  • Waiting time explodes as utilization approaches 100%.
  • A short simulation can estimate averages and later check formulas.

Next: The Anatomy of a Queue