Skip to content
Tutorials Center
My learning

Topics

Lesson 13 of 38 · M/M/1 Queues

M/M/1 Design Trade-offs: Speed, Cost and Service Levels

Use M/M/1 formulas to choose a service rate that balances cost against waiting, and to find the highest arrival rate that still meets a response-time target.

Formulas are most useful when they help you decide something. This lesson uses the M/M/1 results to answer two design questions: how fast should the server be, given what speed costs and what waiting costs? And how much traffic can the system carry while still meeting a response-time promise?

Question 1: how fast should the server be?

A faster server costs more but shortens waits. Suppose speed costs cs per unit of service rate per hour, and each customer-hour spent in the system costs cw (lost sales, staff time, frustration). Then the total cost per hour is

C(μ) = cs μ + cw L = cs μ + cw λ / (μ - λ)

since L = λ/(μ - λ). The first term grows with μ and the second shrinks. Setting the derivative to zero:

cs - cw λ / (μ - λ)² = 0 so μ* = λ + √(cw λ / cs)

The optimal service rate is the arrival rate plus a safety margin that grows with the square root of the traffic. This "square-root" pattern appears again in the staffing lesson.

Example: λ = 8 per hour, cs = 1, cw = 4. Then μ* = 8 + √32 ≈ 13.66. The utilization at the optimum is 8/13.66 ≈ 0.59: spare capacity is cheap compared with waiting.

Python Optimal service rate

Press Run (or Ctrl + Enter) to see the result here.

Program input · what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The cost curve is flat near the minimum, so a rough choice close to μ* loses little, but too slow a server is penalized sharply.

Question 2: how much traffic can we carry under an SLA?

A service-level agreement (SLA) often reads: "at most a fraction α of requests may take longer than t." For M/M/1, P(T > t) = e^(-(μ - λ)t). We need e^(-(μ - λ)t) ≤ α, which rearranges to

μ - λ ≥ ln(1/α) / t so λ_max = μ - ln(1/α) / t

Take a web server with μ = 20 requests per second and a target that 95% of requests finish within 0.5 seconds. Then α = 0.05, ln 20 ≈ 3.0, so we need μ - λ ≥ 3.0 / 0.5 ≈ 6.0 per second, and λ_max ≈ 14 requests per second. The server is only 70% utilized at the limit.

Python Maximum load under an SLA

Press Run (or Ctrl + Enter) to see the result here.

Program input · what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The tighter the promise (smaller t or α), the more spare capacity you must keep. When ln(1/α)/t exceeds μ, no traffic at all can meet the target: the server is simply too slow for that promise, and λ_max is 0.

Faster server or more customers per server?

Doubling the service rate μ has a strong effect. At fixed λ = 8, μ = 10 gives W = 0.5 hours, but μ = 20 gives W = 1/12 hours, a 6-fold drop, not a 2-fold drop. Speed improvements pay off most when utilization is high.

Python What does doubling the speed buy?

Press Run (or Ctrl + Enter) to see the result here.

Program input · what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

These are averages from a model. Real service times are not exactly exponential, so use formulas to size the system roughly, then confirm with measurement or simulation before committing money.

Check your understanding

Quick check · 4 questions

of 4 answered You scored / 4 · perfect!

  1. 1. In the cost model C(mu) = csmu + cwlambda/(mu - lambda), why is there an optimal mu?

    A faster server costs more but cuts waiting; the total has a minimum in between.
  2. 2. mu = 10 per minute. Which lambda gives P(T > 1 minute) = e^(-4)?

    The tail is e^(-(mu - lambda) t), so mu - lambda = 4 and lambda = 6.
  3. 3. Making an SLA stricter (smaller alpha) means...

    lambda_max = mu - ln(1/alpha)/t decreases when alpha gets smaller.
  4. 4. With lambda = 8, going from mu = 10 to mu = 20 changes W from 0.5 to what?

    W = 1/(20 - 8) = 1/12, a 6-fold reduction.

Summary

  • Total cost cs μ + cw λ/(μ - λ) is minimized at μ* = λ + √(cw λ / cs).
  • To meet "P(T > t) ≤ α", you need μ - λ ≥ ln(1/α)/t.
  • Strict targets force low utilization.
  • Speed-ups pay off most when the server is already busy.

Next: Simulation Check of M/M/1