Skip to content
Tutorials Center
My learning

Topics

Lesson 37 of 38 Ā· Projects and Practice

Practice: Variance Reduction

Practice problems on antithetic variates, control variates, importance sampling and stratified sampling, each with a worked, runnable solution.

Reading about variance reduction is easy; knowing which trick to use is the skill. This practice set gives you four problems built around methods from the chapter. Try each yourself before opening the solution. Each solution runs, so you can change the numbers and experiment.

The common thread is the approach to measure a method: repeat the whole estimator many times, then compare the spread (standard deviation) of the estimates at equal cost.

Problem 1: estimating π

The integral āˆ«ā‚€Ā¹ 4/(1 + x²) dx equals Ļ€. Estimate it with N = 1000 function evaluations in three ways: plain Monte Carlo, antithetic variates, and a control variate g(x) = x (known mean 1/2). Report the standard deviation of each over 2,000 repetitions.

Hint: the integrand is decreasing, so antithetic pairs should help. The control variate x is strongly (negatively) correlated with 4/(1 + x²).

Python Solution 1: three estimators of pi

Press Run (or Ctrl + Enter) to see the result here.

Program input Ā· what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Problem 2: a rare tail

Estimate P(Z > 5) for a standard normal Z; the exact value is about 2.87 Ɨ 10⁻⁷. With 100,000 plain samples you would expect only about 0.03 hits. Use importance sampling with a normal proposal centered at 5, and report the estimate with its standard error.

Hint: the weight for proposal N(μ, 1) is exp(āˆ’Ī¼x + μ²/2).

Python Solution 2: importance sampling for P(Z > 5)

Press Run (or Ctrl + Enter) to see the result here.

Program input Ā· what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Problem 3: stratifying π

Estimate π with the same integrand as Problem 1, using N = 100 points stratified into 100 equal strata (one draw each). Compare its standard deviation with plain sampling over 2,000 repetitions, and explain why the ratio is so large.

Hint: a smooth function is almost flat inside a thin stratum.

Python Solution 3: stratified sampling

Press Run (or Ctrl + Enter) to see the result here.

Program input Ā· what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The stratified spread is tiny because, within each stratum of width 0.01, the integrand changes very little, so almost all variance (the variation between strata) is removed.

Problem 4: combining methods

Estimate E[e^{Z}] where Z is standard normal. The exact value is e^{1/2} ā‰ˆ 1.6487. Use antithetic variates (Z and āˆ’Z) combined with a control variate Z² (known mean 1). Compare variances against plain sampling at equal cost.

Hint: first form the pair average (e^Z + e^{āˆ’Z})/2, which is cosh(Z); then regress it on Z², since cosh(z) ā‰ˆ 1 + z²/2 near zero.

Python Solution 4: antithetic plus control variate

Press Run (or Ctrl + Enter) to see the result here.

Program input Ā· what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

How to choose a method

Situation First thing to try
Output is monotone in the inputs Antithetic variates
A related quantity has a known mean Control variates
The event of interest is rare Importance sampling
The integrand is smooth and low dimensional Stratified sampling

Always check your variance-reduced estimator against a plain one (or an exact value) on a small problem first. A bug in the weights or the control's known mean gives a confident, narrow, and wrong answer.

Check your understanding

Quick check Ā· 3 questions

of 3 answered You scored / 3 Ā· perfect!

  1. 1. You want P(Z > 6) for a standard normal. Which method is the best first try?

    The event is extremely rare, so you must sample the tail directly and reweight.
  2. 2. A control variate must have:

    The known mean keeps the estimator unbiased; correlation produces the variance reduction.
  3. 3. How should you compare two estimators fairly?

    A single run cannot show spread; equal cost makes the comparison honest.

Summary

  • Match the method to the structure of the problem: monotone, known-mean companion, rare event, or smooth integrand.
  • Methods combine: antithetic pairs plus a control variate often beat either alone.
  • Always judge an estimator by repeated runs at equal cost, and verify on a problem with a known answer.

Next: Practice: MCMC and Simulation of Systems