Skip to content
Tutorials Center
My learning

Topics

Lesson 18 of 47 ยท Monte Carlo Methods

Random Numbers and Sampling

Learn how pseudo-random generators work, why seeds matter, and how to sample exponential, Gaussian and direction distributions by inverse transform, Box-Muller and rejection.

Every Monte Carlo program starts with a stream of random numbers, and almost every physics problem needs them shaped into a specific distribution: exponential lifetimes for radioactive decay, Gaussian velocities for a gas, uniform directions for a scattered neutron. This lesson shows where the numbers come from and the three standard recipes for turning uniform numbers into anything else.

Left: the cumulative distribution of the exponential law with arrows turning uniform values into x values. Right: a histogram of 20,000 samples matching the exact density
Inverse-transform sampling: draw u uniformly, then solve F(x) = u. The histogram of the results follows the target density.

What you will learn

  • How a pseudo-random generator works and why seeds make runs repeatable.
  • The inverse-transform method, with exponential lifetimes as an example.
  • The Box-Muller transform for Gaussian numbers.
  • Rejection sampling when the inverse is not available.
  • How to pick random directions uniformly on a sphere.

Key terms

PRNG: a pseudo-random number generator, a deterministic algorithm with an internal state.

Seed: the initial state; the same seed gives the same stream.

CDF F(x): the probability that the variable is at most x.

PDF f(x): the probability density, f = dF/dx.

Uniform deviate u: a random number uniformly distributed in [0, 1).

Pseudo-random generators

A computer cannot toss a coin; it computes numbers that pass statistical tests of randomness. The classic example is the linear congruential generator (LCG):

x_{n+1} = (a x_n + c) mod m, u_n = x_n / m

With well chosen a, c, m the sequence looks random but repeats after at most m numbers (the period). Poor parameters give short periods and visible patterns; the infamous RANDU generator from the 1960s put triples of numbers on a small number of planes. NumPy's default generator, np.random.default_rng(), uses the PCG64 algorithm: a huge period (2ยนยฒโธ) and excellent statistics. Always use it rather than the older global np.random.rand.

Python A tiny LCG, good and bad

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Seeds. default_rng(42) always returns the same stream, which makes bugs reproducible and published results checkable. For parallel runs use np.random.SeedSequence(42).spawn(k) to create k statistically independent child streams; never seed several workers with consecutive integers by hand and assume they are independent.

Inverse-transform sampling

Given a target with CDF F(x), draw u uniformly and return x = Fโปยน(u). Why does it work? The probability that x โ‰ค a equals the probability that u โ‰ค F(a), which is F(a): exactly the CDF we wanted.

Example: radioactive decay. A nucleus with mean lifetime ฯ„ has the lifetime density f(t) = (1/ฯ„) e^(โˆ’t/ฯ„) and CDF F(t) = 1 โˆ’ e^(โˆ’t/ฯ„). Solving F(t) = u gives

t = โˆ’ฯ„ ln(1 โˆ’ u)

Since 1 โˆ’ u is also uniform, many programs use t = โˆ’ฯ„ ln u. The figure above shows the idea for ฯ„ = 1: each uniform u on the vertical axis maps through the curve to a lifetime on the horizontal axis.

Python Sampling decay lifetimes

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Gaussian numbers: Box-Muller

The Gaussian CDF has no elementary inverse, so use a different trick. Take two independent uniforms uโ‚, uโ‚‚:

zโ‚ = โˆš(โˆ’2 ln uโ‚) cos(2ฯ€ uโ‚‚), zโ‚‚ = โˆš(โˆ’2 ln uโ‚) sin(2ฯ€ uโ‚‚)

Both zโ‚ and zโ‚‚ are independent standard normal numbers (mean 0, variance 1). The proof works in polar coordinates: the radius squared of a 2D Gaussian follows an exponential law (which you can sample by inverse transform), and the angle is uniform. To get a Gaussian of mean ฮผ and standard deviation ฯƒ, use ฮผ + ฯƒ z. In practice, call rng.normal(ฮผ, ฯƒ, size); it uses a faster algorithm with the same result.

Rejection sampling

When you can evaluate the density f(x) but cannot invert its CDF, use rejection sampling: pick a simple envelope, throw darts, keep those under the curve.

  1. Pick M with f(x) โ‰ค M on the interval [a, b].
  2. Draw x uniformly on [a, b] and y uniformly on [0, M].
  3. Accept x if y โ‰ค f(x), otherwise discard it.

The accepted x follow f. The efficiency is the area under f divided by the area of the rectangle, so a tight envelope wastes fewer samples.

Python Box-Muller and rejection sampling

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Random directions on a sphere

A neutron scatters in a random direction. A tempting recipe, "pick the polar angle ฮธ uniformly in [0, ฯ€] and the azimuth ฯ† uniformly in [0, 2ฯ€]", is wrong: it piles points near the poles, because the ring at polar angle ฮธ has circumference 2ฯ€ sin ฮธ. The correct recipe draws cos ฮธ uniformly in [โˆ’1, 1] (this is Archimedes' hat-box theorem: equal heights cut equal areas on a sphere).

Python Uniform directions: right and wrong

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Common misconceptions

"If the numbers look random, any transformation of them is fine." The ฮธ-uniform sphere example shows that a plausible recipe can be biased. Verify distributions with a histogram and known moments.

"Seed with the clock to be safe." For research, record the seed so the result can be reproduced.

"rng.random() can return 1.0." It returns values in [0, 1): never exactly 1. That matters for log(1 โˆ’ u) (safe) versus log(u) (could be log(0), though the chance is negligible).

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What does the inverse-transform method compute?

    Solve F(x) = u for x. The resulting x has CDF F.
  2. 2. For exponential lifetimes with mean tau, a sample is

    Inverting F(t) = 1 - exp(-t/tau) gives t = -tau ln(1 - u).
  3. 3. Why is picking theta uniformly in [0, pi] wrong for directions on a sphere?

    A ring at polar angle theta has circumference 2 pi sin(theta). Draw cos(theta) uniformly instead.
  4. 4. In rejection sampling, what does the acceptance rate depend on?

    A tight envelope rejects fewer points.
  5. 5. How should you generate independent random streams for parallel workers?

    spawn produces statistically independent child streams.

Summary

  • PRNGs are deterministic; seed them for reproducibility and use np.random.default_rng.
  • Inverse transform: x = Fโปยน(u); for decay, t = โˆ’ฯ„ ln(1 โˆ’ u).
  • Box-Muller turns two uniforms into two Gaussians; rejection sampling needs only the density.
  • Uniform directions need cos ฮธ uniform, not ฮธ.
  • Always check a sampler with a histogram and known moments.

Next: Monte Carlo Integration