Skip to content
Tutorials Center
My learning

Topics

Lesson 8 of 38 · Basics and Little's Law

Utilization and Stability

Understand traffic intensity, utilization and why a queue is stable only when rho < 1. Simulate unstable queues and size the number of servers.

A queue can only settle into a steady state if the servers can, on average, keep up with the work arriving. This simple idea is captured by a number called the utilization or traffic intensity. This lesson defines it, shows what goes wrong when it reaches 1, and uses it to size a system.

Offered load and utilization

Each arriving customer brings 1/μ units of work (time of one server). If λ customers arrive per unit time, the work arrives at the rate λ/μ. This quantity is called the offered load, denoted a. It is measured in erlangs, named after the Danish engineer Agner Erlang, and it equals the average number of servers that would be busy if there were enough of them.

With c servers, each can supply one unit of work per unit time, so the capacity is c. The utilization is

ρ = λ / (c μ) = a / c

For one server ρ = λ/μ. If a call center receives 45 calls per hour, each takes 6 minutes (μ = 10 per hour), then a = 4.5 erlangs. With 5 agents, ρ = 4.5/5 = 0.9.

The stability condition

For a queue with unlimited waiting room, the system is stable if and only if ρ < 1 (for the usual models). If ρ > 1, work arrives faster than the servers can remove it: the queue grows forever and waits grow without limit. If ρ = 1 exactly, random fluctuations still push the queue to drift away, so waits also grow without limit, only more slowly.

Think of the queue length as a random walk with a drift. When customers arrive faster than they leave, the drift is upward. Even when the drift is slightly negative the walk wanders far before returning, which is why waits become huge near ρ = 1.

Watching instability

We simulate three single-server queues with μ = 1 and λ = 0.9, 1.0 and 1.1, and print the average waiting time of consecutive blocks of 10,000 customers. For a stable queue the block averages level off, with noise (the first block is lower because the queue starts empty). For an unstable queue they keep climbing.

Python Stable versus unstable

Press Run (or Ctrl + Enter) to see the result here.

Program input · what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

For λ = 1.1, the average wait of the block keeps rising: the later customers wait longer than the earlier ones, with no end. In a real system, customers would balk or the system would fail, but in the idealized model with unlimited room the wait grows without bound.

Utilization is not wasted capacity

A common mistake is to aim for 100% utilization. Because of randomness, high utilization means long waits, and we will see exactly how long. Typical design targets are well below 1:

  • Call centers often aim for roughly 80-90% occupancy of agents.
  • Web servers are commonly kept below about 70% of capacity to absorb spikes.
  • Emergency services intentionally run low utilization so that an urgent request finds a free unit.

The right target depends on how costly waiting is compared with idle servers.

Sizing the number of servers

The stability condition gives a hard lower bound: you need c > a. A target on utilization gives a practical choice: the smallest c with a / c ≤ ρ_max. Later we will use waiting-time targets instead, which is more accurate.

Python Servers for a utilization target

Press Run (or Ctrl + Enter) to see the result here.

Program input · what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Notice that going from a 90% to an 80% target costs one additional server here, and 70% costs another. Each step buys much shorter waits.

When a queue has several servers, remember that ρ is the load per server. A system with c = 10 and ρ = 0.9 behaves very differently from one with c = 1 and ρ = 0.9; the larger system is more efficient at the same utilization. We study this in the multi-server chapter.

Check your understanding

Quick check · 4 questions

of 4 answered You scored / 4 · perfect!

  1. 1. Calls arrive at 30 per hour and each takes 4 minutes. What is the offered load in erlangs?

    mu = 15 per hour, so a = 30 / 15 = 2 erlangs (also 30 per hour times 4/60 hour).
  2. 2. With 4 servers and offered load 3.2 erlangs, what is the utilization?

    rho = a / c = 3.2 / 4 = 0.8.
  3. 3. What happens in a queue with unlimited room when rho is greater than 1?

    Work arrives faster than it can be served, so the backlog grows forever.
  4. 4. Why do many designers avoid utilization near 100%?

    Waiting time grows sharply as rho approaches 1, so spare capacity protects service quality.

Summary

  • Offered load a = λ/μ (erlangs) is the average number of busy servers if capacity were unlimited.
  • Utilization ρ = a/c is the load per server.
  • A queue with unlimited room is stable only if ρ < 1.
  • Waits explode as ρ nears 1, so design for spare capacity.

Next: Birth-Death Processes