Skip to content
Tutorials Center
My learning

Topics

Lesson 4 of 36 ยท Getting Started

Classifying Stochastic Processes

A map of stochastic processes: discrete or continuous time and state, Markov, stationary, independent increments and martingales, with a Python check.

There are far more stochastic processes than anyone can study one by one, so mathematicians sort them by a few key properties. Knowing the categories tells you which tools apply. This lesson is a map of the whole track.

First sort: time and state

Two yes/no questions give four families.

Discrete state Continuous state
Discrete time Markov chain, simple random walk autoregressive series, Gaussian random walk
Continuous time Poisson process, birth-death process Brownian motion, diffusions

Chapters 3 and 4 treat discrete time with discrete states. Chapters 5 and 6 treat continuous time with discrete states. Chapter 7 reaches continuous states.

Second sort: the dependence structure

How does the future depend on the past? This is the most important question.

Independent processes

The values are independent. Repeated coin flips are an example. There is nothing to predict beyond the distribution itself.

Markov processes

The future depends on the past only through the present. Formally,

Text
P(X_(n+1) = y | X_n = x, X_(n-1) = x_(n-1), ..., X_0 = x_0) = P(X_(n+1) = y | X_n = x)

The current state is all you need. The random walk is Markov: knowing where it stands now, the path that led there does not help predict the next step.

Martingales

A martingale is a "fair game" process: E[X_(n+1) | X_0, ..., X_n] = X_n. Your expected fortune after the next round equals your fortune now. The symmetric random walk is both Markov and a martingale, but the two ideas are different: a martingale restricts the mean of the next value, while the Markov property restricts which information matters.

Stationary processes

A process is strictly stationary if shifting time leaves its joint distributions unchanged: (X_(t1), ..., X_(tk)) has the same law as (X_(t1+h), ..., X_(tk+h)) for every shift h. Think of a system that has been running so long that it looks the same whenever you start watching. This matters in Chapter 4 when we study long-run behavior.

Processes with independent increments

Random walks, Poisson processes and Brownian motion all add up independent pieces. They have independent increments, and usually stationary ones too.

flowchart TD
    A["Stochastic processes"] --> B["Markov"]
    A --> C["Martingales"]
    A --> D["Stationary"]
    A --> E["Independent increments"]
    E -->|"most of them are"| B

Properties overlap: one process can belong to several classes.

Examples sorted by property

Process Markov? Martingale? Independent increments?
Symmetric random walk yes yes yes
Walk with up-probability 0.6 yes no (drifts upward) yes
Weather (sun / rain), modeled as a chain yes no no
Running maximum of the walk, alone no no no

The running maximum is not Markov by itself: to predict whether it will increase you need to know the current position as well as the maximum. Pairing them, (position, maximum), restores the Markov property. This trick of adding information to the state is common.

A small empirical check

You cannot prove a property by simulation, but you can spot violations. Here we test whether the sign of the next step of a walk depends on the previous step. For a true random walk, both conditional frequencies should be near 1/2.

Python Does the past step help predict the next one?

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Now a process whose steps do depend on the past: each step repeats the previous one with probability 0.8.

Python A sticky process: the next step depends on the last

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The steps here form a two-state Markov chain (a topic for Chapter 3). The two conditional probabilities are far from equal, so the past clearly helps.

Where the rest of the track goes

  • Chapters 3 and 4: Markov chains and their long-run behavior.
  • Chapters 5 and 6: Poisson processes and continuous-time Markov chains.
  • Chapter 7: random walks and Brownian motion.
  • Chapters 8 and 9: applications, simulation and projects.

Check your understanding

Quick check ยท 3 questions

of 3 answered You scored / 3 ยท perfect!

  1. 1. The Markov property says that the future, given the past and present, depends on...

    Given the present state, the past carries no extra information about the future.
  2. 2. A martingale satisfies...

    A martingale is a fair game: the conditional expectation of the next value equals the current value.
  3. 3. Which process has both independent and stationary increments?

    The walk adds independent, identically distributed steps, so its increments are independent and stationary.

Summary

  • Processes are sorted by time (discrete or continuous) and state (discrete or continuous).
  • The Markov property says the present state is all that matters for the future.
  • Martingales model fair games; stationary processes look the same under time shifts.
  • Independent, stationary increments define random walks, Poisson processes and Brownian motion.
  • Adding information to the state can restore the Markov property.

Next: Probability and Expectation Review