Skip to content
Tutorials Center
My learning

Topics

Lesson 35 of 46 ยท Stochastic Models and Simulation

Chapter 7 review: stochastic models and simulation

Review Monte Carlo simulation, random walks, Markov chains and queues with a one-picture summary, a formula table, a quick experiment and a ten-question quiz.

This chapter taught you to let chance into your models and to read the answer as a distribution. Here is the whole chapter on one page: a poster, a table of formulas, a short experiment on the law of large numbers, and a ten-question quiz.

Four cards summarising Monte Carlo, random walks, Markov chains and queues.
Chapter 7 on one page: four stochastic tools and what to remember about each.

Recap table

Tool Key idea Formula or rule
Monte Carlo average many random runs error โ‰ˆ ฯƒ/โˆšN
Random walk sum of independent steps spread โˆšn, mean 0
Gambler's ruin reach N before 0 fair game: k/N
Markov chain next state depends on now ฯ€ยทP = ฯ€, rows sum to 1
M/M/1 queue random arrivals and service ฯ = ฮป/ฮผ, W_q = ฯ/(ฮผ โˆ’ ฮป)
Little's law links queue size and time L = ฮปยทW

Habits of a good simulation

  • Seed it so a colleague can reproduce your run, and try other seeds to see the noise.
  • Run enough replications to make the standard error small relative to the effect you care about.
  • Report a distribution: mean, a percentile and the chance of a bad outcome, not just one number.
  • Validate with a case where an exact answer exists (the birthday formula, M/M/1).
  • Question the inputs. The outputs can be no better than the distributions you fed in.

Try it

This experiment shows the law of large numbers: the running average of die rolls settles toward 3.5, with fluctuations shrinking like 1/โˆšn.

Python The running average of a die

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

A second check: what the load does to a queue. The table gives the M/M/1 average wait for several loads with a service time of 1 minute, so you can see where waits become unreasonable.

Python Wait times by load

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Check your understanding

Quick check ยท 10 questions

of 10 answered You scored / 10 ยท perfect!

  1. 1. A Monte Carlo estimate has an error of 0.10 with 1,000 samples. Roughly how many samples give an error of 0.01?

    A tenfold error reduction needs about 100 times more samples.
  2. 2. A symmetric random walk of 2,500 steps has a typical distance from the start of about:

    sqrt(2500) = 50.
  3. 3. In a fair gambler's ruin game starting with 2 coins and aiming for 8, the chance of success is:

    k / N = 2 / 8 = 0.25.
  4. 4. The stationary distribution pi of a Markov chain satisfies:

    A stationary distribution is unchanged by one more step of the chain.
  5. 5. Each row of a transition matrix sums to:

    From any state the chain must go somewhere, so the probabilities add to 1.
  6. 6. For M/M/1 with lambda = 0.9 and mu = 1 per minute, the average time in the system W is:

    W = 1 / (mu - lambda) = 1 / 0.1 = 10 minutes.
  7. 7. Why do queues form even when average capacity exceeds average demand?

    Variability creates waits even at loads below 1.
  8. 8. Which does a larger number of simulation runs improve?

    More runs reduce sampling noise only.
  9. 9. Which statement about the Markov property is correct?

    That is the definition of the Markov property.
  10. 10. Which is a good way to validate a simulation?

    Matching known results builds trust that the code is right.

Summary

  • Stochastic models produce distributions; report spread and tail risk, not only means.
  • Monte Carlo error falls like 1/โˆšN; random walks spread like โˆšn.
  • Markov chains settle to ฯ€ = ฯ€P; M/M/1 queues blow up as ฯ โ†’ 1.
  • Validate with an exact special case and treat the inputs with suspicion.

Next: Chapter 8 overview: model validation and communication