Skip to content
Tutorials Center
My learning

Topics

Lesson 44 of 57 ยท Bayesian Statistics and Simulation

Monte Carlo Simulation

Use Monte Carlo simulation to estimate probabilities, expectations and integrals, check the law of large numbers and quantify error, with seeded Python code.

Some probability questions are hard to solve with pencil and paper: What is the chance of getting a pair or better in a poker hand? How long will a customer queue be? What is the probability that a project with ten uncertain tasks finishes on time? Monte Carlo simulation answers them by brute force: let the computer run the random process many times and count what happens.

The recipe is always the same:

  1. Describe the random process as code.
  2. Repeat it N times, recording the outcome each time.
  3. Estimate the probability as the fraction of runs where the event occurred (or the expectation as the average).
flowchart LR
  A["Model the random process"] --> B["Simulate N times"]
  B --> C["Record the outcome each run"]
  C --> D["Average or count"]
  D --> E["Estimate plus its error"]

A first simulation: two dice

What is the probability that two fair dice sum to 8? Exactly, it is 5/36 โ‰ˆ 0.1389. By simulation: roll many pairs and count. Always seed the random generator so results are reproducible.

Python Estimating a dice probability

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The estimates settle towards 0.1389 as N grows. That is the law of large numbers at work: averages of many independent runs converge to the true expectation.

How accurate is the estimate?

A simulated probability pฬ‚ is itself a random quantity. Its standard error is โˆš(pฬ‚(1 โˆ’ pฬ‚)/N). Quadrupling N halves the error, so each extra digit of accuracy costs 100 times more runs. For an average of values the standard error is s/โˆšN. This also tells you how many runs you need: to estimate a probability near 0.1 within ยฑ0.001 (95%), you need about 1.96ยฒ ร— 0.1 ร— 0.9 / 0.001ยฒ โ‰ˆ 346,000 runs.

Python Estimate with a margin of error

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Simulating a game: the birthday problem

How many people must be in a room before two share a birthday with probability above 50%? Intuition says about 183; the answer is only 23. Simulating makes the surprise believable.

Python The birthday problem by simulation

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Estimating an integral: the value of pi

Monte Carlo is not only for probabilities. Throw random points into the unit square; the fraction that land inside the quarter circle xยฒ + yยฒ โ‰ค 1 estimates its area ฯ€/4. More generally, the average of f(X) for random X approximates the integral of f, which is a powerful way to handle integrals in many dimensions.

Python Estimating pi with random points

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Simulating uncertainty in a real decision

Suppose a project has three tasks done one after another, each with a random duration. A single "expected total" hides the risk. Simulation shows the whole distribution of totals and the chance of missing a deadline.

Python Project completion risk

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Adding the "most likely" values gives 21 days, yet the chance of finishing within 21 is far below 50% because the durations are skewed to the right. Monte Carlo reveals what a single-number plan cannot.

Good practice

  • Seed the generator and report the seed; never trust a single run without an error estimate.
  • Vectorize with NumPy instead of Python loops; it is often 100 times faster.
  • Check your simulation against a case with a known exact answer (like the dice) before trusting it on a hard one.
  • Remember that simulation reflects your model: if the assumptions (independence, distributions) are wrong, the answer will be precisely wrong.
  • Variance reduction tricks (antithetic variables, importance sampling) get the same accuracy with fewer runs; they are worth learning for large problems.

Simulation is also the way Bayesians explore posterior distributions, as in Bayesian Inference with the Beta-Binomial Model, and the basis of the resampling methods in the next lesson.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. A Monte Carlo estimate of a probability uses N = 10,000 runs. To halve its standard error you need about how many runs?

    Standard error is proportional to 1 over the square root of N, so halving it requires four times as many runs.
  2. 2. Why seed the random number generator?

    A fixed seed produces the same pseudo-random stream, so you and others can reproduce the exact numbers.
  3. 3. In the pi simulation, why multiply the fraction of points inside the quarter circle by 4?

    Fraction inside is approximately (pi/4) / 1, so pi is about 4 times that fraction.
  4. 4. Which law guarantees that the average of many simulated outcomes approaches the true expectation?

    The law of large numbers says sample averages converge to the expected value as the number of independent trials increases.

Summary

  • Monte Carlo simulation estimates probabilities and expectations by repeating a random process many times.
  • Accuracy improves with โˆšN; report a standard error or margin of error.
  • Simulation can estimate integrals, solve counter-intuitive puzzles and quantify project or financial risk.
  • Seed your generator, vectorize, and validate against known answers.
  • Results are only as good as the model assumptions.

Next: Bootstrap and Permutation Tests