Skip to content
Tutorials Center
My learning

Topics

Lesson 62 of 149 ยท Modules and the Standard Library

The statistics Module

Summarize small data sets with Python's statistics module. Compute mean, median, mode, standard deviation, quantiles, correlation and linear regression.

Before you reach for pandas or NumPy, remember that Python ships with a module for everyday statistics. The statistics module answers questions like "what is the average order?", "how spread out are these test scores?" and "do these two columns move together?" with plain function calls and no installation.

It's designed for small to medium data sets that fit in memory. It trades speed for accuracy and simplicity: results are computed carefully, with exact arithmetic where possible. For millions of numbers, use NumPy or pandas instead. In this lesson you'll cover averages, spread, quantiles, relationships between two variables, and NormalDist.

Measures of the center

The three classic "averages" are mean, median and mode, and they can tell different stories about the same data:

Python mean, median and mode

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The mean is the sum divided by the count, and a single outlier (250) drags it far up. The median, the middle value after sorting, ignores how extreme the extremes are, which makes it the better summary for incomes, house prices and response times. The mode is the most common value, and multimode() returns all values tied for most common.

flowchart TB
    D["Your data"] --> C["Center<br/>mean, median, mode"]
    D --> S["Spread<br/>variance, stdev, quantiles"]
    D --> R["Relationship<br/>correlation, regression"]

The module's functions fall into three families: where the data sits, how far it spreads, and how two series relate.

mean() keeps the type it can: integers give an exact result when possible, and fractions or decimals stay exact. fmean() converts everything to float and is the fast choice for plain numbers. Related functions include median_low(), median_high(), median_grouped() and geometric_mean() and harmonic_mean() for rates and ratios.

Measures of spread

Two data sets can share a mean and still look nothing alike. Spread tells you how far values typically stray from it:

Python Variance and standard deviation

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Both sets average 50, but the standard deviation reveals the difference. The module offers two versions of each measure:

  • pstdev / pvariance: for a population, when your list is every item you care about.
  • stdev / variance: for a sample, when your list is a subset used to estimate a bigger group. They divide by n - 1 instead of n.

When in doubt about whether data is a sample, it usually is, so use stdev(). Variance is the standard deviation squared, which is why it's in squared units.

Quantiles and the five-number summary

quantiles() cuts the data into equal-sized groups. With the default n=4 you get the three quartile cut points:

Python Quartiles

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The interquartile range (Q3 - Q1) measures the spread of the middle half of the data and is robust against outliers. The default method is "exclusive", suited to samples. Use "inclusive" when your data is the entire population.

Relationships between two variables

Given two equally long series, correlation() tells you how strongly they move together, from -1 (opposite) through 0 (unrelated) to +1 (perfectly in step). linear_regression() fits the straight line that best predicts one from the other:

Python Correlation and regression

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Here, each extra hour of study is associated with about 4.4 extra points. Remember the old rule: correlation is not causation. Two series can move together because of a third factor, or by coincidence. Pass proportional=True to linear_regression() to force the line through the origin, and method="ranked" to correlation() for Spearman's rank correlation.

NormalDist: working with the bell curve

statistics.NormalDist models a normal (Gaussian) distribution. You can build one from data and ask probability questions:

Python NormalDist

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

cdf() gives the probability of a value at or below x; inv_cdf() goes the other way. zscore() says how many standard deviations a value sits from the mean. Arithmetic on a NormalDist shifts or scales it.

Errors to expect

The module raises StatisticsError when a question has no answer, such as the mean of an empty list or the standard deviation of a single value. Catch it, or check the length first:

Python StatisticsError

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Data also needs to be numbers. statistics won't convert strings for you, so clean text input first, as in the CSV lesson.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Which average is least affected by one extreme outlier?

    The median depends only on the middle position after sorting, so one very large or small value barely changes it.
  2. 2. When should you use stdev() instead of pstdev()?

    stdev() is the sample version (dividing by n - 1) and estimates the spread of a larger population. pstdev() treats the data as the whole population.
  3. 3. A correlation of -0.95 between two series means

    Values near -1 show a strong inverse relationship. It says nothing about cause and effect.
  4. 4. What does statistics.quantiles(data, n=4) return?

    To split data into n groups you need n - 1 cut points, so n=4 returns Q1, the median and Q3.

Summary

  • mean(), fmean(), median(), mode() and multimode() describe the center; the median resists outliers.
  • stdev() and variance() are for samples, pstdev() and pvariance() for whole populations.
  • quantiles() returns cut points for quartiles, deciles and more.
  • correlation(), covariance() and linear_regression() describe how two series relate; correlation is not causation.
  • NormalDist answers probability questions about normal distributions, and StatisticsError signals data that is too small or empty.

Next: Files, where you'll read and write data on disk.