Skip to content
Tutorials Center
My learning

Topics

Lesson 28 of 49 ยท Random Numbers and Statistics

Probability Distributions and Sampling

Sample from binomial, Poisson, exponential, beta, gamma and multivariate normal distributions with NumPy, plot histograms, see the CLT and bootstrap.

A probability distribution describes how likely each outcome is. Coin flips follow a binomial distribution, waiting times between events often follow an exponential one, and measurement errors tend to follow the famous bell curve. NumPy's Generator can draw samples from dozens of distributions, and a histogram of a large sample is the quickest way to see what a distribution looks like.

This lesson builds on Random Numbers with Generators: you create one rng = np.random.default_rng(seed) and call a method per distribution. You will also see two classic ideas that rely on sampling, the central limit theorem and the bootstrap.

Discrete distributions

A discrete distribution gives whole-number outcomes.

  • rng.binomial(n, p, size): the number of successes in n independent trials, each succeeding with probability p.
  • rng.poisson(lam, size): how many events happen in a fixed interval when they occur at an average rate lam, such as emails per hour.
  • rng.geometric(p, size): how many trials until the first success.
Python Binomial and Poisson samples

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

For a Poisson distribution the mean and the variance are both equal to lam, a handy check that the sample is behaving.

Continuous distributions

A continuous distribution gives any real number in a range.

Method Typical use
rng.normal(loc, scale) measurements with symmetric noise
rng.exponential(scale) waiting time between random events (scale is the mean)
rng.beta(a, b) probabilities and proportions, always between 0 and 1
rng.gamma(shape, scale) positive skewed quantities such as rainfall or durations
rng.lognormal(mean, sigma) positive values with a long right tail, such as incomes
Python Summary statistics of several distributions

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The sample means land very close to the theoretical values noted in the comment. With more samples they get closer still: that is the law of large numbers.

Seeing a distribution with histograms

A histogram counts how many samples fall in each bin. np.histogram returns the counts and bin edges; matplotlib's plt.hist draws them directly. Use density=True to scale the bars so the total area is 1.

Python Histograms of four distributions

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

flowchart LR
    G["Generator<br/>rng"] --> D["pick a method:<br/>normal, binomial, beta ..."]
    D --> S["array of samples"]
    S --> H["np.histogram or plt.hist"]
    S --> M["mean, std, percentiles"]

Draw samples, then summarize them with a histogram or statistics.

Correlated samples with multivariate_normal

rng.multivariate_normal(mean, cov, size) draws vectors whose coordinates are related. The mean is a vector and the covariance matrix says how the coordinates vary together: the diagonal holds the variances and the off-diagonal entries the covariances. NumPy builds this on the SVD (see Eigenvalues, SVD and Decompositions).

Python Two correlated variables

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The central limit theorem

One of the most useful facts in statistics: if you average many independent values, the averages follow a bell curve, even when the individual values do not. Let us average draws from a very non-bell-shaped distribution, the exponential.

Python Central limit theorem in action

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The averages are centered on the same mean, but their spread shrinks by a factor of sqrt(40) and the shape becomes symmetric. That square-root shrinkage is why more data gives more certain estimates.

Bootstrap sampling

How confident can you be in a statistic computed from one small dataset? The bootstrap answers by pretending your sample is the whole population. Resample it with replacement many times, compute the statistic each time, and look at how much it varies.

flowchart TD
    D["Original sample (n values)"] --> R["Resample n values<br/>with replacement"]
    R --> S["Compute the statistic<br/>(for example the mean)"]
    S --> L["Repeat thousands of times"]
    L --> I["Spread of the results =<br/>uncertainty of the statistic"]

The bootstrap estimates uncertainty by resampling the data you already have.

Python Bootstrap confidence interval for a mean

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Notice how fancy indexing (Fancy Indexing) builds all 10,000 resamples in one step, with no Python loop. The bootstrap works for statistics that have no simple formula, such as the median or a correlation.

More distributions: a tour of the rest

Besides the normal, binomial and Poisson distributions, a handful of others cover most modelling needs. Each one is a method of the Generator, and each is described by a few parameters. In the table, "mean" is the theoretical average you can use to sanity-check a sample.

Method Models Parameters Theoretical mean
rng.exponential(scale, size) waiting time between events scale = average wait (1/rate) scale
rng.geometric(p, size) trials until the first success p = success probability 1 / p
rng.gamma(shape, scale, size) waiting time for shape events shape, scale shape * scale
rng.beta(a, b, size) a probability or proportion in (0, 1) a, b a / (a + b)
rng.chisquare(df, size) sum of df squared standard normals df degrees of freedom df
rng.lognormal(mean, sigma, size) positive, right-skewed values (incomes, sizes) mean and sigma of the log exp(mean + sigma**2 / 2)
Python Sampling each distribution and checking the mean

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Notice the ranges: exponential, gamma, chi-square and log-normal values are always positive; beta values stay between 0 and 1; geometric values are whole numbers starting at 1.

Seeing the shapes

A text histogram is enough to recognise the shapes without a plotting library. The function below prints one bar per bin.

Python Text histograms of the shapes

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

How they relate

These families are connected, which helps you remember them:

  • The exponential is a gamma with shape 1, and a gamma is a sum of exponentials.
  • The geometric is the discrete cousin of the exponential: trials instead of continuous time.
  • The chi-square with df degrees of freedom is a gamma with shape df / 2 and scale 2.
  • Dividing independent gamma samples gives a beta: X / (X + Y) with X ~ gamma(a) and Y ~ gamma(b).
  • A Poisson count of events per hour goes together with an exponential waiting time between events.
Python Checking the relationships

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Pick a distribution by asking what the number represents: a waiting time (exponential, gamma), trials to success (geometric), a proportion (beta), a positive skewed size (log-normal) or a sum of squares (chi-square, used in statistical tests).

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Which method samples the number of successes in n independent trials?

    binomial(n, p) counts successes in n trials with success probability p. poisson counts events in an interval.
  2. 2. What does the central limit theorem say about averages of many independent values?

    Means of many independent draws become approximately normal, with a spread that shrinks like 1/sqrt(n).
  3. 3. In the bootstrap, how are the resamples drawn from the original data?

    Resampling with replacement mimics drawing new samples from the population, so values can repeat within a resample.
  4. 4. In multivariate_normal(mean, cov), what do the off-diagonal entries of cov control?

    The diagonal holds variances. Off-diagonal entries are covariances, which create correlation between coordinates.

Summary

  • Each distribution is a method on the generator: binomial, poisson, exponential, beta, gamma, normal, multivariate_normal and many more, all taking size.
  • Discrete distributions give counts; continuous ones give real numbers.
  • Check samples with mean, std and histograms (np.histogram, plt.hist(density=True)).
  • multivariate_normal draws correlated vectors using a covariance matrix.
  • Averages of many draws become bell-shaped (central limit theorem), with spread shrinking like 1/sqrt(n).
  • The bootstrap resamples with replacement to estimate uncertainty for any statistic.

Next, summarize data with Statistics, Correlation and Covariance.