Skip to content
Tutorials Center
My learning

Topics

Lesson 57 of 68 ยท Probability and Statistics for Engineers

Poisson and Renewal Processes

Model random arrivals over time: the Poisson process, exponential inter-arrival times, superposition, thinning and renewal counting, with simulations.

The earlier lessons gave you a Poisson distribution for a count in a window and an exponential distribution for a waiting time. They are actually two views of one object: a random process in which events (failures, packets, customers, particle hits) occur at random moments along a time axis. This lesson builds that process, shows the link between its counts and its gaps, and then relaxes the one assumption that makes the Poisson case so tidy, which leads to renewal processes.

You should be comfortable with Binomial, Poisson and Other Discrete Distributions and with the exponential distribution from Probability and Random Variables.

Counting processes

Let N(t) be the number of events that have occurred up to time t. It starts at N(0) = 0, only ever steps up by one, and so it is a staircase. The same process can be described in two equivalent ways: by the counts N(t) in any time window, or by the inter-arrival times T1, T2, ..., the gaps between consecutive events. The arrival time of the n-th event is the sum S_n = T1 + ... + Tn.

flowchart LR
    A["Inter-arrival times<br/>T1, T2, T3, ..."] -- "add up" --> B["Arrival times<br/>S1, S2, S3, ..."]
    B -- "count how many<br/>fall in [0, t]" --> C["Counting process<br/>N(t)"]
    C -- "differences between<br/>jumps" --> A

Gaps, arrival times and counts carry the same information.

The Poisson process

A Poisson process with rate lambda (events per unit time) is the counting process with these properties:

  1. N(0) = 0.
  2. Independent increments: counts in disjoint time intervals are independent.
  3. Stationary: the count in any interval of length t depends only on t.
  4. In a very short interval h, there is one event with probability about lambda h, and two or more events with negligible probability.

From these, the number of events in any interval of length t is Poisson with mean lambda t:

P(N(t) = k) = exp(-lambda t) (lambda t)**k / k!

You can see where it comes from: chop [0, t] into n tiny slices, each with success probability lambda t / n. The count is binomial with n p = lambda t, and the binomial becomes Poisson when n grows, the rare-event limit from the previous lesson.

The gaps T1, T2, ... are independent exponential variables with mean 1 / lambda. A short derivation: the first event comes after time t exactly when there are no events in [0, t], so P(T1 > t) = P(N(t) = 0) = exp(-lambda t), the exponential survival function. Because of independent increments, the process restarts afresh after each event, so every gap has the same law. This is the memoryless property in action: whatever happened before, the wait for the next event looks the same.

Python Build a Poisson process from exponential gaps

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Generating only exponential gaps and counting the arrivals that land in a two-minute window gives a count with mean near 6, variance near 6 (the Poisson fingerprint), and probabilities close to the Poisson PMF. The simulated and exact columns agree to within the sampling noise of 10,000 runs.

Merging and thinning

Two properties make the Poisson process easy to use in system design.

Superposition. Merge independent Poisson streams with rates lambda1 and lambda2 and you get a Poisson process with rate lambda1 + lambda2. Calls arriving from two exchanges form one stream with the summed rate.

Thinning. Keep each event of a Poisson process independently with probability p (a packet survives a lossy link; a defect is detected by an inspector) and the kept events form a Poisson process with rate p lambda, independent of the discarded ones.

Python Gap statistics, memorylessness, merging and thinning

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The mean gap is close to 0.25, the tail probability matches exp(-2) = 0.1353, and the remaining wait after already waiting 0.25 is again about 0.25. Merging gives a rate of about 10 and thinning a rate of about 1, with mean gaps near 1 (the small differences from the exact values are simulation noise).

When the gaps are not exponential: renewal processes

The Poisson process assumes memoryless gaps, which suits independent random arrivals but not many machines. A pump that is replaced when it fails and then lasts roughly the same time again, a bus that runs close to schedule, or a maintenance cycle with a fixed overhaul interval all have gaps that are not exponential.

A renewal process keeps the key idea (independent, identically distributed gaps T1, T2, ...) but allows any positive distribution for them, with mean mu and variance sigma**2. After each event, the process "renews": it starts over statistically. The Poisson process is the special case with exponential gaps.

Three results are worth knowing.

  • Long-run rate. N(t) / t approaches 1 / mu: events per unit time is the reciprocal of the mean gap. This is the elementary renewal theorem. Components that last 500 hours on average are replaced about once every 500 hours, whatever the shape of the lifetime distribution.
  • Spread of the counts. For large t, N(t) is approximately normal with mean t / mu and variance sigma**2 t / mu**3. Regular gaps (small sigma) give far steadier counts than exponential gaps (where sigma = mu).
  • Renewal function. m(t) = E[N(t)] is the expected count. It satisfies the renewal equation, an integral equation that combines the gap density with m itself; it is solved numerically in practice, or by Laplace transforms (see The Laplace Transform).
Python Same mean gap, different regularity

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

In 100 minutes both processes average about 50 events, but the exponential gaps give a count variance near 52 (variance-to-mean ratio about 1, as for any Poisson count) while the regular gaps give a variance near 5.4, which matches the theory value sigma**2 t / mu**3 = 5.56. Same average, very different predictability. For capacity planning that difference is the difference between a half-empty queue and an overflowing one.

When the rate changes: the nonhomogeneous Poisson process

A homogeneous Poisson process has one constant rate lam. Real arrival streams rarely do: a web server is busy at noon and quiet at 4 a.m., a call centre peaks after lunch, radioactive samples decay away. A nonhomogeneous Poisson process (NHPP) keeps the useful structure but lets the rate depend on time, lam(t).

The rules are the same with one change. Events in disjoint intervals are independent, and the number of events in (s, t] is Poisson with a mean equal to the integrated rate:

m(s, t) = integral from s to t of lam(u) du, P(N(t) - N(s) = k) = e**(-m) * m**k / k!.

The function Lambda(t) = integral from 0 to t of lam(u) du is the mean value function. The waiting time to the next event is no longer exponential with a fixed mean; instead, given the last event at time s, the chance of no event before t is exp(-m(s, t)).

To simulate an NHPP, use thinning (Lewis and Shedler): pick a constant lam_max that is at least as large as lam(t) everywhere, generate a fast homogeneous process at rate lam_max, and keep each point at time t with probability lam(t) / lam_max. The kept points follow the wanted rate.

Python A daily traffic pattern: expected counts and thinning

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

With a rate that swings between 2 and 18 per hour across the day, the three-hour mid-morning window is far busier than the first three hours after midnight, even though both windows have the same length. An NHPP is the right model whenever the shape of the rate matters for capacity planning.

When to use which

  • Arrivals from many independent sources (calls, web requests, cosmic rays) are usually well modelled by a Poisson process. Check that the count variance is close to the mean.
  • Replacement of components, cyclic machines, scheduled services: use a renewal process with a realistic gap distribution (gamma and Weibull are common).
  • If the rate changes over time (a daily traffic pattern), use a non-homogeneous Poisson process with rate lambda(t), where the mean count in [a, b] is the integral of lambda(t) from a to b: calculus again.

Common slips

  • Mixing up the rate lambda (events per time) with the mean gap 1 / lambda; in scipy, expon(scale=...) takes the mean gap.
  • Assuming independence of arrivals when events come in bursts. A count variance well above the mean is the warning sign.
  • Using the Poisson process for components that wear out: a worn part is more likely to fail the longer it has run, which contradicts memorylessness.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Events form a Poisson process with rate 5 per hour. How is the count in a 3-hour window distributed?

    The count in an interval of length t is Poisson with mean lambda t = 5 x 3 = 15.
  2. 2. What is the distribution of the time between consecutive events of a Poisson process with rate lambda?

    P(T > t) = P(no events in [0, t]) = exp(-lambda t), the exponential survival function.
  3. 3. Two independent Poisson streams with rates 2 and 7 per minute are merged. What is the merged process?

    Superposition of independent Poisson processes is Poisson with the sum of the rates.
  4. 4. A renewal process has mean gap 8 hours. What is the long-run number of events per hour?

    By the elementary renewal theorem, N(t)/t tends to 1/mu = 1/8, regardless of the shape of the gap distribution.

Summary

  • A counting process N(t) can be described by counts or by inter-arrival times.
  • A Poisson process has independent, stationary increments; counts in a window of length t are Poisson with mean lambda t and gaps are independent exponentials with mean 1 / lambda.
  • Merging independent Poisson streams adds their rates, and random thinning multiplies the rate by the keep probability.
  • A renewal process allows any gap distribution; the long-run rate is 1 / mu, and the count variance for large t is about sigma**2 t / mu**3.
  • Regular gaps give steadier counts than exponential gaps with the same mean.

Next: Statistics, Curve Fitting and Uncertainty.