Skip to content
Tutorials Center
My learning

Topics

Lesson 55 of 68 ยท Probability and Statistics for Engineers

Binomial, Poisson and Other Discrete Distributions

Learn the binomial, geometric, Poisson and hypergeometric distributions: PMFs, means, variances, when each applies, and scipy.stats code for engineers.

Many engineering questions count things. How many of 20 sampled bolts are out of tolerance? How many trials until a test fails? How many calls reach a switch in a minute? How many bad chips are in a handful drawn from a batch? The quantity is a whole number, so the tool is a discrete distribution, a probability mass function (PMF) that gives P(X = k) for each count k.

Four distributions cover most of these cases. The skill is not the formulas but recognising the situation: what is being counted, and is each trial independent? This lesson builds on the notation from Random Variables, Covariance and Correlation and the scipy.stats interface from Probability and Random Variables.

Bernoulli trials and the binomial distribution

A Bernoulli trial has two outcomes, success (probability p) and failure (1 - p): a part passes or fails, a packet arrives or is lost. Repeat it n times with independent trials and the same p, and let X count the successes. Then X is binomial with parameters n and p:

P(X = k) = C(n, k) p**k (1 - p)**(n - k), for k = 0, 1, ..., n

Here C(n, k) = n! / (k! (n - k)!) counts the arrangements of k successes among n trials. The mean and variance are

E[X] = n p, Var[X] = n p (1 - p)

You can see why: X is the sum of n independent Bernoulli variables, each with mean p and variance p (1 - p), and means and variances of independent variables add.

The geometric distribution

Keep running the same independent trials, but now count the trials until the first success. This is geometric:

P(X = k) = (1 - p)**(k - 1) p, for k = 1, 2, 3, ...

with E[X] = 1 / p and Var[X] = (1 - p) / p**2. It is the discrete twin of the exponential distribution, and like it, it is memoryless: if the first 5 trials all failed, the number of additional trials needed has the same distribution as at the start.

Python Binomial and geometric with scipy.stats

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

In a sample of 20, the chance of exactly two defective bolts is about 0.285 and the chance of at most two is about 0.677. The "at most" probabilities use the CDF (cdf), while "more than" uses the survival function (sf), which is 1 - cdf and more accurate for small tail probabilities. The ppf line answers a design question: with 99% confidence the sample will hold no more than 6 defective bolts. On the geometric side, you wait ten trials on average for the first defect, but with a large variance of 90: the average is a poor guide to any single run. The last line shows memorylessness, with both ratios equal to 0.4783.

The Poisson distribution

Suppose events arrive at random but at a steady average rate: calls at a switch, defects along a cable, photons on a detector. Count them in a fixed window. If the average count in the window is mu, the count is Poisson:

P(X = k) = exp(-mu) mu**k / k!, for k = 0, 1, 2, ...

Its mean and variance are both mu. That equality is a quick diagnostic: if your count data have a variance much larger than the mean, the Poisson model is probably wrong (events are clustering).

The Poisson distribution is the limit of the binomial when n is large and p is small with n p = mu held fixed: many tiny chances, rarely successful. This is called the law of rare events. The next lesson, Poisson and Renewal Processes, places the Poisson count in a process that unfolds over time.

The hypergeometric distribution

Binomial sampling assumes independent trials, which holds when you sample with replacement, or from a huge population. Take a small batch and sample without replacement, and each draw changes the odds for the next. A batch of M items contains K bad ones; you draw N at random; X is the number of bad ones you find. This is hypergeometric:

P(X = k) = C(K, k) C(M - K, N - k) / C(M, N)

with mean N K / M, the same as the binomial with p = K / M, but a smaller variance, N (K/M) (1 - K/M) (M - N) / (M - 1). The factor (M - N) / (M - 1) is the finite-population correction; it approaches 1 when the sample is tiny compared with the batch. Acceptance sampling in quality control is the classic use.

Python Poisson, the rare-event limit and the hypergeometric

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The Poisson probabilities are 0.0498 for a silent minute and 0.1008 for exactly five calls, and a minute with more than six calls happens only about 3.3% of the time. The loop shows the binomial probability of two successes converging on the Poisson value 0.22404 as n grows. In the hypergeometric check, inspecting 8 of 50 chips with 5 bad finds at least one bad chip with probability 0.5985, higher than the 0.5695 you would get if you wrongly treated the draws as independent: without replacement you cannot keep drawing the same good chip, so you learn more per sample. Note that scipy.stats.hypergeom takes the population size M, the number of bad items n and the sample size N, in that order.

Choosing the right distribution

flowchart TD
    A["Counting something discrete"] --> B{"Fixed number of<br/>trials n?"}
    B -- "yes" --> C{"Sampling without<br/>replacement from a<br/>small batch?"}
    C -- "yes" --> H["Hypergeometric"]
    C -- "no" --> Bi["Binomial(n, p)"]
    B -- "no" --> D{"Counting trials until<br/>the first success?"}
    D -- "yes" --> G["Geometric(p)"]
    D -- "no" --> P["Events in a time or<br/>space window: Poisson(mu)"]

A quick decision guide. Check the independence assumption before trusting any of them.

Distribution Counts Mean Variance
Binomial(n, p) successes in n trials n p n p (1 - p)
Geometric(p) trials until first success 1 / p (1 - p) / p**2
Poisson(mu) events in a window mu mu
Hypergeometric(M, K, N) bad items in a sample without replacement N K / M N (K/M)(1 - K/M)(M - N)/(M - 1)

Watch the geometric convention: scipy counts the trial on which the success happens (starting at 1), as above. Some books count failures before the first success (starting at 0), which shifts the mean to (1 - p) / p.

Seeing the shapes

Python Plot three PMFs

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The binomial is a symmetric-looking hump around n p = 4.2, the Poisson is slightly skewed with a longer right tail, and the geometric always peaks at 1 and decays by a constant factor 1 - p per step.

Common slips

  • Using a binomial when the trials are not independent or p changes between trials. Check the process, not just the numbers.
  • Treating P(X <= k) and P(X < k) as the same. For discrete variables they differ by P(X = k).
  • Using Poisson for counts whose variance is much bigger than the mean.
  • Sampling without replacement from a small batch and still using the binomial.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. A fair test is repeated 12 times with independent outcomes and each fails with probability 0.25. Which distribution describes the number of failures?

    A fixed number of independent trials with a constant success (here failure) probability gives a binomial distribution. Its mean is 12 x 0.25 = 3.
  2. 2. What is the mean number of trials until the first success when p = 0.2?

    The geometric mean is 1/p = 5 trials, counting the successful trial itself.
  3. 3. For a Poisson distribution with mu = 4, what is the variance?

    A Poisson variable has equal mean and variance, both mu.
  4. 4. You inspect 5 items drawn without replacement from a batch of 30 that holds 6 bad items. Which model is exact?

    Sampling without replacement from a small finite batch changes the odds on each draw, which the hypergeometric distribution accounts for.

Summary

  • Binomial: successes in n independent trials; mean n p, variance n p (1 - p).
  • Geometric: trials until the first success; mean 1 / p; memoryless.
  • Poisson: events in a window at a steady average rate; mean equals variance; the rare-event limit of the binomial.
  • Hypergeometric: sampling without replacement from a finite batch; same mean as the binomial but smaller variance.
  • In scipy.stats, use pmf, cdf, sf, ppf, mean and var; use sf for "more than" tails.

Next: Poisson and Renewal Processes.