Skip to content
Tutorials Center
My learning

Topics

Lesson 36 of 76 ยท Statistics for Data Science

Probability and Distributions You Will Meet

Meet the normal, binomial, Poisson, exponential, uniform and long-tailed distributions, and see the law of large numbers and central limit theorem in simulations.

Every statistical method rests on a picture of how data come about. A probability distribution is that picture: it says which values a random quantity can take and how likely each one is. When you know the shape that fits your data, you can answer questions that raw numbers cannot: how many failures to expect this week, how long a customer will wait, how unusual a result is, or how much a sample average will wiggle.

You do not need all of probability theory for data science. A handful of distributions cover most of what you will meet, and two big ideas, the law of large numbers and the central limit theorem, explain why averages and confidence intervals work at all. This lesson introduces both through simulation: you generate data with a known distribution, so you can see the shapes yourself. The SciPy course goes deeper into descriptive statistics and distributions; here we focus on which distribution to reach for and why.

Random variables and probability in a minute

A random variable is a number produced by a process with some randomness: the number of visitors tomorrow, the time until a package arrives. It is discrete if it takes separate values (counts: 0, 1, 2, ...) and continuous if it can take any value in a range (times, weights).

  • For discrete variables, the probability mass function (PMF) gives the probability of each value, and the probabilities add up to 1.
  • For continuous variables, the probability density function (PDF) is a curve whose area over an interval is the probability of landing in that interval. The height of the curve at a point is not itself a probability.
  • The cumulative distribution function (CDF) gives the probability of being at or below a value, for both kinds.

Useful rules: probabilities lie between 0 and 1; for independent events, multiply (both heads on two flips: 0.5 x 0.5 = 0.25); for mutually exclusive events, add. Simulation lets you check such answers without algebra: repeat the process many times and count.

flowchart TD
    R["Random variable"] --> D["Discrete<br/>(counts)"]
    R --> C["Continuous<br/>(measurements)"]
    D --> B["Binomial<br/>(successes out of n)"]
    D --> P["Poisson<br/>(events per interval)"]
    C --> N["Normal<br/>(sums, measurement noise)"]
    C --> E["Exponential<br/>(time between events)"]
    C --> U["Uniform<br/>(all equally likely)"]
    C --> L["Lognormal, Pareto<br/>(long right tails)"]

The distributions you will meet most often, grouped by the kind of quantity they describe.

The normal distribution

The normal (Gaussian) distribution is the bell curve, defined by its mean and standard deviation. It appears when a quantity is the sum of many small independent effects, such as measurement errors, or heights within a group. About 68% of values fall within one standard deviation of the mean, 95% within two, and 99.7% within three.

The z-score (x - mean) / std tells you how many standard deviations a value is from the mean, which makes different scales comparable.

Python The normal distribution and the 68-95-99.7 rule

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The three functions you will use constantly are pdf (density), cdf (probability below a value, with sf = 1 - cdf for "above") and ppf (the inverse of the CDF: the value at a given percentile). Every SciPy distribution has them.

Binomial: successes out of n tries

If you run n independent trials, each succeeding with probability p, the number of successes follows a binomial distribution. Typical uses: how many of 200 emails get opened, how many of 50 patients respond to a treatment, how many conversions among 1,000 visitors. The mean is n * p and the standard deviation is sqrt(n * p * (1 - p)).

Conditions: a fixed number of trials, only two outcomes, the same p every time, and independence. When these fail (customers influencing each other, p varying by segment), the binomial understates the variation.

Python Binomial: conversions among 200 visitors

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Seeing 15 conversions or more has a probability of about 8%, so a day with 15 conversions is not strong evidence that the true rate has changed from 5%. This kind of calculation is the heart of hypothesis testing.

Poisson and exponential: events in time

Many data science quantities are counts of events in a fixed interval: support tickets per hour, defects per meter of cable, goals per match. If events are independent and happen at a constant average rate, the count follows a Poisson distribution with one parameter, the rate. Its mean and variance are both equal to the rate, which gives a quick check: if the variance of your counts is much larger than the mean (overdispersion), plain Poisson is too simple.

The exponential distribution is the partner: the time between events in a Poisson process. It is continuous, skewed to the right, and "memoryless": having waited ten minutes already does not change how much longer you will wait.

Python Poisson counts and exponential waiting times

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The mean and the variance of the counts are close to each other, as the Poisson model predicts. The exponential gaps average about 15 minutes, but the median is lower: most gaps are short and a few are very long.

Uniform: all equally likely

The uniform distribution gives every value in a range the same density. It models a fair die, a random number between 0 and 1, or something about which you know only the limits. In practice its main role is as a source of randomness: percentiles of any variable are uniformly distributed, which is why random sampling and many simulations start from uniform numbers. It is also a handy prior assumption when you know nothing but bounds.

Long-tailed distributions

Not everything is bell-shaped. Sales per customer, file sizes, city populations and the number of followers are long-tailed: most values are small, and a few are enormous. Typical models are the lognormal (the logarithm is normal; arises when effects multiply) and the Pareto (a power law; extreme tails). Two consequences matter in practice:

  • The mean is dominated by the rare giants and is unstable; sample means jump around a lot. Use medians, percentiles or logs.
  • "Typical" and "total" differ: a small share of customers can produce most of the revenue.
Python Lognormal data look normal after a log transform

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Taking logs is one of the most useful tricks in data science: models and tests that assume roughly normal data work much better on the log scale.

The law of large numbers

The law of large numbers says that as you collect more independent observations, their average settles toward the true mean. It is why a casino profits over thousands of bets and why estimates from larger samples are more reliable. It does not say that short-run luck "evens out": after five heads in a row, the next flip is still 50-50.

Python Running mean of die rolls approaching 3.5

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The early average swings widely, then narrows around the true mean of 3.5. How fast it narrows is quantified by the standard error, which shrinks with the square root of the sample size.

The central limit theorem

The central limit theorem (CLT) is the reason the normal distribution is everywhere. It says: take many independent observations from almost any distribution (with finite variance) and compute their average; the distribution of that average approaches a normal distribution as the sample size grows, even when the original data are not normal. Its spread is the original standard deviation divided by the square root of the sample size.

This is why you can build confidence intervals and tests for means without knowing the exact shape of the data. It also has limits: with extremely skewed or heavy-tailed data, you may need a very large sample for the approximation to be good.

Python Averages of skewed data become bell-shaped

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

For a single observation (n = 1) the histogram is the skewed exponential shape. By n = 50, the averages are nearly bell-shaped (the skewness has dropped from 2.1 to 0.3), and the measured spread matches the theoretical 10 / sqrt(n).

Skewness and kurtosis: describing the shape

Mean and standard deviation tell you where the data sit and how wide they are. Two more numbers describe the shape.

  • Skewness measures asymmetry. It is 0 for a symmetric distribution, positive when the long tail points right (a few very large values, like incomes) and negative when it points left.
  • Kurtosis measures how heavy the tails are compared with a normal distribution. SciPy reports excess kurtosis by default, so a normal distribution scores 0; positive values mean heavier tails and more extreme outliers than normal, negative values mean lighter tails.

Both are sensitive to outliers, so treat them as warnings to look at a histogram rather than as exact measurements. A rough guide: a skewness beyond about +/-1 is strongly asymmetric, and a log transform often fixes right skew.

Python Skewness and kurtosis of different shapes

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The uniform distribution has negative excess kurtosis (light tails, no outliers), the exponential and log-normal are strongly right-skewed with heavy tails, and the t distribution is symmetric but heavy-tailed. Check these two numbers whenever you plan to use a method that assumes normality; see Univariate Analysis for how they fit the first look at a column.

More distributions you will meet

Beyond the normal, binomial, Poisson, exponential, uniform and log-normal distributions above, a few more families appear all the time in statistics, because they are the sampling distributions of test statistics or models for proportions and waiting times.

Distribution Typical use SciPy object
t mean of a small sample with unknown variance; t-tests and confidence intervals stats.t(df)
chi-square sum of squared standard normals; tests of variance and of counts stats.chi2(df)
F ratio of two variances; ANOVA stats.f(dfn, dfd)
gamma positive waiting times, rainfall, claim sizes stats.gamma(a, scale=...)
beta a proportion between 0 and 1; click-through rates stats.beta(a, b)
geometric number of trials until the first success stats.geom(p)
log-normal positive, right-skewed values: incomes, house prices stats.lognorm(s, scale=...)

Each SciPy distribution has the same methods, which makes them easy to learn once: pdf (or pmf for discrete ones), cdf, sf (the upper tail, 1 - cdf), ppf (the inverse of cdf, used for critical values) and rvs (random samples).

Python One interface, many distributions

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Notice how the t distribution's critical value shrinks toward the normal's 1.96 as the degrees of freedom grow. That is why a t-test and a z-test agree for large samples.

Seeing the shapes

Python Plotting the density curves

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

When you are unsure which distribution to use, work backward from what the variable measures, as the chooser below this section does. Most real data are only approximately one of these; the families are a vocabulary for describing data and for the tests you meet in Hypothesis Testing.

Choosing a distribution

If the quantity is... Try
a sum or average, or symmetric noise Normal
successes out of a known number of trials Binomial
a count of events per interval Poisson
time between events Exponential
bounded, with no further knowledge Uniform
positive, right-skewed, multiplicative Lognormal (or Pareto for extreme tails)

A distribution is a model, not a fact. Check it with a histogram, a QQ plot or the simple diagnostics above, and stay alert to violations of the assumptions (independence, constant rate).

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Which distribution models the number of successes out of 50 independent trials with the same success probability?

    A fixed number of independent yes/no trials with constant probability is the definition of a binomial setup.
  2. 2. A count variable has mean 4 and variance 20. What does that suggest?

    A Poisson variable has variance equal to its mean. A much larger variance indicates extra variation, such as varying rates across units.
  3. 3. What does the central limit theorem state?

    The theorem is about the distribution of sample averages, not of the raw data. Its spread is the data's standard deviation divided by the square root of n.
  4. 4. After 5 heads in a row with a fair coin, what is the probability of heads on the next flip?

    Flips are independent. The law of large numbers describes long-run averages, not a correction of past luck.
  5. 5. Order values are heavily right-skewed. Which step often helps before applying methods that assume roughly normal data?

    For lognormal-like data, the logarithm is roughly normal, which makes means, standard deviations and linear models behave better.

Summary

  • A distribution describes which values a random quantity takes and how likely they are; PDF/PMF, CDF and percentile functions answer practical questions.
  • Normal: sums and noise. Binomial: successes out of n. Poisson: counts per interval. Exponential: waiting times. Uniform: equal likelihood. Lognormal and Pareto: long right tails.
  • Long tails make means unstable; use medians, percentiles and log transforms.
  • The law of large numbers: bigger samples have averages closer to the truth.
  • The central limit theorem: averages become approximately normal as the sample grows, which is why intervals and tests for means work.

Next: Estimation and Confidence Intervals, turning these ideas into honest statements about uncertainty.