Skip to content
Tutorials Center
My learning

Topics

Lesson 10 of 38 · Sampling Methods

Inverse Transform Sampling

Turn uniform random numbers into exponential, Weibull and custom distributions with the inverse CDF method, verified by histograms and KS tests.

Generators give you uniform numbers. Real simulations need waiting times that are exponential, heights that are normal, losses that follow a heavy-tailed law. Inverse transform sampling is the cleanest way to turn uniform numbers into almost any distribution, as long as you can invert its cumulative distribution function.

The cumulative distribution function

For a random variable X, the cumulative distribution function (CDF) is

Text
F(x) = P(X ≤ x)

It rises from 0 (far left) to 1 (far right) and never decreases. If X has a density f, then F is its running area.

The key fact

Let U be uniform on (0, 1) and let F be a continuous, strictly increasing CDF with inverse F⁻¹. Then

Text
X = F⁻¹(U)  has CDF F.

Why? P(F⁻¹(U) ≤ x) = P(U ≤ F(x)) = F(x), because for a uniform U, P(U ≤ t) = t. So feed a uniform number through the inverse CDF and out comes a sample from the target distribution.

flowchart LR
  A["U uniform on (0,1)"] --> B["solve F(x) = U"]
  B --> C["X = inverse CDF of U"]
  C --> D["X has the target distribution"]

Inverse transform: pick a height on the CDF axis, read across to the x value.

Example: the exponential distribution

The exponential distribution with rate λ models waiting times between random events. Its CDF is F(x) = 1 − e^(−λx) for x ≥ 0. Solving u = 1 − e^(−λx) for x:

Text
x = −ln(1 − u) / λ

Since 1 − U is also uniform, we can use x = −ln(U)/λ; either works. The mean of this distribution is 1/λ.

Python Exponential samples by inversion

Press Run (or Ctrl + Enter) to see the result here.

Program input · what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The histogram follows the exact curve λe^(−λx). We drew from a distribution without any special built-in function, using only uniform numbers and a logarithm.

Checking a sampler: the Kolmogorov–Smirnov test

How do you confirm a sampler is right? Compare the sample's empirical CDF with the true CDF. The Kolmogorov–Smirnov (KS) statistic is the largest gap between them. SciPy's kstest returns it with a p-value.

Python Verify with a KS test

Press Run (or Ctrl + Enter) to see the result here.

Program input · what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Another example: a custom density

Suppose a random variable on [0, 1] has density f(x) = 3x². Its CDF is F(x) = x³, so F⁻¹(u) = u^(1/3). Samples cluster near 1, as the density predicts.

The Weibull distribution (used for lifetimes) has CDF F(x) = 1 − exp(−(x/s)^k), inverting to x = s · (−ln(1−u))^(1/k).

Python Custom and Weibull samplers

Press Run (or Ctrl + Enter) to see the result here.

Program input · what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The Weibull mean is s · Γ(1 + 1/k), and for k = 2 that is s × 0.8862… (the Gamma function at 1.5 equals √π/2).

When it works well, and when it doesn't

  • Works well when F⁻¹ has a simple formula (exponential, Weibull, Pareto, logistic, uniform on an interval, triangular).
  • Works for discrete distributions too: compare U with the cumulative probabilities (see the lesson on discrete sampling).
  • Struggles when F⁻¹ has no closed form, as for the normal distribution. You can still invert numerically (SciPy's ppf does this) but it is slower. The next two lessons give alternatives: Box–Muller and rejection sampling.
  • A subtle precision issue: 1 − u can lose accuracy when u is near 1; use −log(u) or log1p(−u) in careful code.

SciPy offers ppf ("percent point function"), the inverse CDF for every standard distribution. stats.norm.ppf(rng.random(n)) is inverse-transform sampling of a normal distribution in one line.

Check your understanding

Quick check · 4 questions

of 4 answered You scored / 4 · perfect!

  1. 1. If U is uniform on (0, 1) and F is a continuous increasing CDF, what is the distribution of F⁻¹(U)?

    P(F⁻¹(U) ≤ x) = P(U ≤ F(x)) = F(x), so the transformed value has CDF F.
  2. 2. For the exponential distribution with rate λ, F(x) = 1 − e^(−λx). Which formula turns a uniform u into a sample?

    Solving u = 1 − e^(−λx) for x gives x = −ln(1 − u)/λ.
  3. 3. A density f(x) = 3x² on [0, 1] has CDF x³. What is the inverse CDF?

    Solving u = x³ gives x = u^(1/3).
  4. 4. When is inverse transform sampling awkward?

    Without an explicit inverse you need numerical inversion or another method such as Box–Muller or rejection sampling.

Summary

  • The CDF F(x) = P(X ≤ x); if U is uniform on (0, 1), then X = F⁻¹(U) has CDF F.
  • Exponential: x = −ln(1−u)/λ; Weibull: x = s(−ln(1−u))^(1/k); density 3x²: x = u^(1/3).
  • Verify a sampler by comparing its histogram with the density and running a KS test.
  • It is easiest when F⁻¹ has a formula; otherwise use numerical inversion, Box–Muller or rejection.

Next: Sampling the Normal Distribution: Box–Muller