Skip to content
Tutorials Center
My learning

Topics

Lesson 1 of 36 ยท Getting Started

What Is a Stochastic Process?

Learn what a stochastic process is: a collection of random variables indexed by time. See coin flips, queues and prices as processes, with runnable Python.

A single random variable describes one uncertain number: the roll of a die, tomorrow's temperature, the height of a stranger. But many real situations unfold over time. A stock price wanders from day to day. Customers arrive at a shop one after another. A gambler's fortune rises and falls with every bet. To describe these we need a whole family of random variables, one for each moment. That family is called a stochastic process.

This track is about the mathematics of such processes. You will meet Markov chains, Poisson processes, random walks and Brownian motion, and you will simulate every one of them in Python. This first lesson gives the definition and builds your intuition with small examples.

The definition

A stochastic process is a collection of random variables {X_t : t in T} defined on the same probability space. The set T is the index set, usually time. The set of values that X_t can take is the state space S.

In words: for every time t, there is a random quantity X_t. Before you look, X_t is uncertain. After you look, you see one number (or one label) from the state space.

Two common choices of index set:

Index set T Name Example
{0, 1, 2, ...} discrete time the daily closing price, the roll number in a dice game
[0, โˆž) continuous time the number of calls received by a phone exchange up to time t

The word "stochastic" comes from a Greek word for aiming or guessing. It means "involving chance". "Random process" means the same thing.

Example 1: coin flips

Flip a fair coin repeatedly. Let X_n = 1 if flip n is heads and X_n = 0 if it is tails. Then X_1, X_2, X_3, ... is a stochastic process with index set {1, 2, 3, ...} and state space {0, 1}. Its members are independent and identically distributed, which is the simplest process there is.

From the flips we can build a more interesting process. Let S_n = X_1 + ... + X_n be the number of heads in the first n flips. Now consecutive values depend on each other: S_(n+1) is either S_n or S_n + 1.

Python A coin-flip process and its running total

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Run it a few times with different seeds. Each run produces one different sequence. That sequence is called a sample path (or realization or trajectory). The process itself is the whole rule that generates such paths.

Two ways to look at a process

A process is a function of two things: the time t and the outcome ฯ‰ of the random experiment. You can fix either one.

  • Fix the time t. You get one random variable X_t. It has a distribution, a mean and a variance. For the coin-flip total, E[S_n] = n/2.
  • Fix the outcome ฯ‰. You get an ordinary function of time, t โ†ฆ X_t(ฯ‰). That is a sample path, one possible history.
Python Many sample paths, one distribution at a fixed time

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The variance n * 1/4 = 12.5 comes from the fact that each flip has variance 1/4 and the flips are independent.

Example 2: more processes from daily life

Process Index set State space
Weather today: sunny, cloudy, rainy days {sunny, cloudy, rainy}
Number of customers in a queue continuous time {0, 1, 2, ...}
Position of a pollen grain in water continuous time R^3
Your bank balance at the end of each month months real numbers
Which web page a visitor is on clicks the set of pages

Notice how different these are. The state space can be finite, countable or continuous. Time can tick in steps or flow smoothly. Much of this course is about organizing that variety.

flowchart LR
    A["Stochastic process"] --> B["Discrete time"]
    A --> C["Continuous time"]
    B --> D["Discrete state<br/>(Markov chain)"]
    B --> E["Continuous state<br/>(random walk on R)"]
    C --> F["Discrete state<br/>(Poisson process)"]
    C --> G["Continuous state<br/>(Brownian motion)"]

Four typical combinations of time and state, and a famous example of each.

Why bother?

A process lets you ask questions about the evolution, not just one snapshot:

  • How long until the gambler is ruined?
  • In the long run, what fraction of days is rainy?
  • How many customers will have arrived by noon?
  • Does the price ever cross a given barrier?

Each of these is a question about the whole path, so a single random variable is not enough.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What is a stochastic process?

    A stochastic process is a family {X_t : t in T} of random variables on one probability space.
  2. 2. If you fix the outcome of the random experiment and let time vary, what do you get?

    Fixing the outcome gives one ordinary function of time, called a sample path, realization or trajectory.
  3. 3. For S_n = X_1 + ... + X_n with fair coin flips (X = 1 for heads), what is E[S_n]?

    Each flip has mean 1/2, and expectation is additive, so E[S_n] = n/2.
  4. 4. The number of calls received by an exchange up to time t, for every real t >= 0, has which index set?

    t ranges over [0, โˆž), so the index set is continuous.

Summary

  • A stochastic process is a family of random variables {X_t : t in T}; T is the index set and the values live in the state space S.
  • Fixing t gives a random variable; fixing the outcome gives a sample path.
  • Time can be discrete or continuous, and the state space can be discrete or continuous.
  • Processes let us study evolution: hitting times, long-run behavior and counts over time.

Next: States, Time and Sample Paths