Skip to content
Tutorials Center
My learning

Topics

Lesson 38 of 39 ยท Projects and Practice

R Project: Simulation, Monte Carlo and the Bootstrap

Learn by simulating in R: estimate pi, solve the birthday problem, build bootstrap confidence intervals, run a permutation test and check what a 95% interval really means.

Some questions are hard to answer with algebra but easy to answer with a computer: "How likely is it that two of 23 people share a birthday?", "How uncertain is this average?", "Could this difference have been produced by shuffling the labels?" The trick is to simulate: generate many random scenarios, count how often the event occurs, and read off the answer. This project builds five small simulations in R and ties them to ideas from probability distributions, functional programming and hypothesis tests.

R is not available on this site, so the code is read-only. Random results depend on R's generator and the seed, so we only quote values that are exact (known from mathematics) and describe the rest as "close to". The Python block at the end runs the same experiments so you can watch them converge.

The toolbox

Every simulation uses the same four moves:

  1. set.seed() once at the top, so that your results are reproducible.
  2. Generate random data with runif(), rnorm(), rbinom(), sample().
  3. Wrap one repetition of the experiment in a function or an expression.
  4. Repeat it many times with replicate() (or vectorize it) and summarize with mean(), quantile() or hist().

The more repetitions, the closer you get to the truth: the error of a simulated probability shrinks like 1 / sqrt(repetitions), so 10,000 runs give roughly two decimal places and 1,000,000 give three.

Simulation 1: estimating pi

Throw random points into the unit square. The share that lands inside the quarter circle of radius 1 is pi / 4:

R
set.seed(2026)
n <- 1e5
x <- runif(n)
y <- runif(n)
inside <- x^2 + y^2 <= 1
4 * mean(inside)            # close to 3.14

inside is a logical vector; its mean() is the proportion of TRUE. With 100,000 points the estimate is typically within about 0.01 of 3.14159. It is a poor way to compute pi, but a clear example of Monte Carlo integration: any area or average that is hard to compute can be estimated by sampling.

Simulation 2: the birthday problem

In a room of 23 people, what is the probability that at least two share a birthday (ignoring leap days)? Most people guess about 6 percent. The exact answer comes from multiplying the chances that each new person avoids all earlier birthdays:

R
1 - prod((365:343) / 365)        # 0.5072972

365:343 has 23 terms: the chance that the second person differs is 364/365, the third 363/365, and so on. The probability of a shared birthday is just over 50 percent. Now simulate it:

R
one_room <- function(n_people) {
  birthdays <- sample(365, n_people, replace = TRUE)
  anyDuplicated(birthdays) > 0         # TRUE if any value appears twice
}

set.seed(1)
mean(replicate(10000, one_room(23)))    # close to 0.507

Run it for other group sizes with sapply() and plot probability against group size to see the S-shaped curve; it passes 90 percent at 41 people and 99 percent at 57:

R
sizes <- 2:60
probs <- sapply(sizes, \(k) 1 - prod((365:(365 - k + 1)) / 365))
plot(sizes, probs, type = "l", xlab = "People in the room", ylab = "P(shared birthday)")
abline(h = 0.5, lty = 2)

The line crosses the dashed 50 percent line between 22 and 23 people.

Simulation 3: the bootstrap

How uncertain is the average mileage of the 32 cars in mtcars? The sample mean is 20.09 mpg. The textbook t-interval is 17.92 to 22.26. The bootstrap gets an interval without a formula: pretend your sample is the population, resample it with replacement many times, and look at how the mean varies.

R
set.seed(7)
x <- mtcars$mpg
boot_means <- replicate(10000, mean(sample(x, replace = TRUE)))

quantile(boot_means, c(0.025, 0.975))     # close to the t-interval 17.92 to 22.26
sd(boot_means)                            # close to the standard error 6.027 / sqrt(32) = 1.065
hist(boot_means, breaks = 40)             # a bell-shaped histogram around 20.09

Each resample has the same size as the original and is drawn with replacement, so some cars appear twice and some not at all. The middle 95 percent of the bootstrap means is a percentile confidence interval. It works for statistics that have no simple formula, such as the median, a trimmed mean, a correlation or a ratio, just by replacing mean with the statistic. Use more repetitions for tail percentiles, and be careful with very small samples: the bootstrap cannot invent information the sample does not contain.

Simulation 4: a permutation test

In the mtcars project, manual cars averaged 7.24 mpg more than automatic cars. A permutation test asks: if transmission were unrelated to mileage, how often would randomly shuffling the labels produce a gap this large?

R
mpg <- mtcars$mpg
am  <- mtcars$am
observed <- mean(mpg[am == 1]) - mean(mpg[am == 0])      # 7.24494

set.seed(11)
shuffled <- replicate(10000, {
  fake <- sample(am)                                      # shuffle the labels
  mean(mpg[fake == 1]) - mean(mpg[fake == 0])
})
mean(abs(shuffled) >= abs(observed))                      # permutation p-value

Shuffling breaks any real link between am and mpg and builds the null distribution from the data themselves. The share of shuffles with a gap at least as extreme as the observed one is the two-sided p-value. It will be tiny, in the same region as the t-test's 0.001, and needs no normality assumption.

Simulation 5: what does "95% confidence" mean?

A 95 percent confidence interval is a claim about the procedure: if you repeated the study many times, about 95 percent of the intervals would contain the true value. Check it by simulating studies where you know the truth:

R
set.seed(3)
true_mean <- 50
covered <- replicate(5000, {
  sample_data <- rnorm(25, mean = true_mean, sd = 10)
  ci <- t.test(sample_data)$conf.int
  ci[1] <= true_mean && true_mean <= ci[2]
})
mean(covered)               # close to 0.95

In about 95 percent of the 5,000 simulated studies the interval catches 50; in the other 5 percent it misses. Any single interval either contains the truth or does not; the 95 percent describes how often the method works. Try n = 5 (the t-interval still reaches about 95 percent for normal data) or replace rnorm() by a very skewed distribution, and watch the coverage drop for small samples.

Make the results reliable

  • Use enough repetitions. For a probability near 0.5, 10,000 runs give an error of about 0.005; for rare events (probability 0.001) you need millions or a smarter method.
  • Vectorize when you can. replicate() is a loop in disguise. For the pi example, one vector of 100,000 points is much faster than 100,000 function calls.
  • Report the uncertainty. A simulated estimate has its own error; run twice with different seeds and check that the numbers agree.
  • Keep the seed. Print it in your report so that others can reproduce the numbers.

Run the same experiments in Python

This Python block repeats the five simulations with NumPy and a fixed seed. Python's random numbers differ from R's, so the digits will not match an R session, but they converge to the same exact values.

Python Python comparison: Monte Carlo, birthday, bootstrap, permutation

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Common mistakes

  • Forgetting replace = TRUE in a bootstrap resample.
  • Calling set.seed() inside the replicated function, which repeats the same draw every time.
  • Using too few repetitions and reporting more digits than the error allows.
  • Reading a confidence interval as "a 95 percent chance the truth lies in this particular interval".
  • Using a bootstrap on a tiny sample or on a statistic that depends on extremes (the maximum).

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What is the idea behind the bootstrap?

    Treating the sample as a stand-in for the population lets you estimate sampling variability by resampling.
  2. 2. In a permutation test, what do you shuffle?

    Shuffling the labels creates data where the null hypothesis is true by construction.
  3. 3. What does 95% confidence mean in the coverage simulation?

    The 95% refers to the long-run success rate of the method, not to a single computed interval.
  4. 4. How does the error of a simulated probability change as the number of repetitions n grows?

    Quadrupling the repetitions halves the error, so each extra digit of accuracy costs about 100 times more runs.

Summary

  • Simulation estimates probabilities and uncertainty by repeating a random experiment: set.seed(), generate, wrap in a function, replicate(), summarize.
  • Monte Carlo integration, the birthday problem (exact: 0.5073 for 23 people), the bootstrap, permutation tests and coverage checks all use the same loop.
  • The bootstrap resamples with replacement; a permutation test shuffles labels; both need no formulas.
  • Report the seed and the number of repetitions, and remember the error shrinks only as 1 / sqrt(n).

Next: Practice Exercises with Solutions