Skip to content
Tutorials Center
My learning

Topics

Lesson 2 of 39 · Getting Started

Probability for Physicists

Review the probability you need for statistical mechanics: distributions, averages, variance, the binomial and Gaussian distributions, and why sums of many terms become sharp.

Statistical mechanics is applied probability. You rarely need advanced theory, but you need to be fluent in a handful of ideas: probability distributions, averages, variance, independence and the Gaussian shape that large sums always approach. This lesson collects them with a physics flavor.

Probability distributions and averages

Suppose a system can be in states labeled s = 1, 2, 3, ... and state s occurs with probability p_s. Probabilities are never negative and they must add up to one:

Text
p_s ≥ 0,     Σ_s p_s = 1

If a quantity A takes the value A_s in state s, its average (also called the expectation value or mean) is

Text
⟨A⟩ = Σ_s p_s A_s

Averages are linear: ⟨aA + bB⟩ = a⟨A⟩ + b⟨B⟩, for any constants a and b, whether or not A and B are related. This rule is used constantly. For instance, the average energy of N particles is N times the average energy of one particle, even when they interact.

Variance and standard deviation

The variance measures how spread out a quantity is:

Text
σ_A² = ⟨(A − ⟨A⟩)²⟩ = ⟨A²⟩ − ⟨A⟩²

The standard deviation σ_A is the square root of the variance and has the same units as A. The second form, ⟨A²⟩ − ⟨A⟩², is the one you will use most: compute the average of the square and subtract the square of the average.

Python Average and variance of a die

Press Run (or Ctrl + Enter) to see the result here.

Program input · what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Independent variables

Two quantities are independent if knowing one tells you nothing about the other. For independent A and B:

Text
⟨AB⟩ = ⟨A⟩⟨B⟩,      σ²_(A+B) = σ_A² + σ_B²

The second rule is the heart of why large systems are predictable. If you add N independent, identically distributed contributions each with mean μ and variance σ², the total has

Text
mean = Nμ,     variance = Nσ²,     standard deviation = σ√N

So the relative spread of the total is σ√N / (Nμ) = (σ/μ) / √N. It shrinks as 1/√N. With N ~ 10^23 that is about 3 × 10^-12.

The binomial distribution

Flip N independent coins, each landing heads with probability p. The number of heads n follows the binomial distribution:

Text
P(n) = C(N, n) p^n (1 − p)^(N−n),   where C(N, n) = N! / (n! (N − n)!)

Its mean is Np and its variance is Np(1 − p). This formula describes many physical situations: N gas molecules each in the left or right half, N spins each up or down, N two-level atoms each excited or not.

Python Binomial formula against a simulation

Press Run (or Ctrl + Enter) to see the result here.

Program input · what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The Gaussian shape

For large N the binomial distribution looks like a bell curve, the Gaussian (normal) distribution:

Text
P(x) = (1 / √(2πσ²)) · exp( −(x − μ)² / (2σ²) )

The central limit theorem says that the sum of many independent random contributions (with finite variance) is approximately Gaussian, no matter what the individual distribution looks like. This is why bell curves are everywhere in physics: pressure and energy are sums over a huge number of molecules.

Two facts about the Gaussian are worth remembering:

  • About 68% of the probability lies within μ ± σ and about 95% within μ ± 2σ.
  • The integral ∫ exp(−a x²) dx over all real x equals √(π/a). You will use this when integrating Boltzmann factors.
Python Watch a sum become Gaussian

Press Run (or Ctrl + Enter) to see the result here.

Program input · what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Conditional thinking: equal a priori probabilities

In statistical mechanics the most important probability assignment is usually the simplest: if you know nothing else, every accessible microstate is equally likely. If there are Ω accessible microstates, each has probability 1/Ω. We will elevate this to a postulate in chapter 3. Counting then becomes the main skill, which is why the next lesson is about combinatorics and Stirling's approximation.

Check your understanding

Quick check · 4 questions

of 4 answered You scored / 4 · perfect!

  1. 1. A quantity A has ⟨A⟩ = 3 and ⟨A²⟩ = 11. What is its variance?

    Variance = <A^2> - <A>^2 = 11 - 9 = 2.
  2. 2. N independent, identical contributions each have standard deviation s. What is the standard deviation of their sum?

    Variances of independent variables add, giving N s^2, so the standard deviation is s sqrt(N).
  3. 3. What is the variance of a binomial distribution with N trials and success probability p?

    The variance is N p (1 - p). The mean is N p.
  4. 4. Why do relative fluctuations of macroscopic quantities become negligible for large systems?

    The mean grows as N and the standard deviation as sqrt(N), so the ratio falls as 1/sqrt(N).

Summary

  • Probabilities are non-negative and sum to one; averages are ⟨A⟩ = Σ p_s A_s and are linear.
  • Variance is ⟨A²⟩ − ⟨A⟩²; for independent variables variances add.
  • A sum of N identical independent terms has relative spread of order 1/√N.
  • The binomial distribution has mean Np and variance Np(1 − p), and tends to a Gaussian for large N.
  • Equal probability for every accessible microstate will be our starting point.

Next: Combinatorics and Stirling's Approximation