Skip to content
Tutorials Center
My learning

Topics

Lesson 32 of 76 ยท Visualization and Storytelling

Showing Distributions and Uncertainty

Draw violin plots, ECDFs, error bars and bootstrapped confidence bands in matplotlib, and learn how to show uncertainty honestly in data science charts.

A bar chart of group averages is the most common chart in business reports, and one of the most misleading. It shows one number per group and hides everything else: how spread out the values are, whether the groups overlap heavily, whether one group is made of two clusters, and how much the average itself would change if you collected the data again. A reader sees two bars with different heights and concludes the groups differ. Often they do not, or not by much.

This lesson shows how to draw what the average hides. First the distribution of the values (box, violin and ECDF plots, with raw points when there are few), then the uncertainty of an estimate (error bars, bootstrapped intervals and confidence bands). You will need the ideas from Looking at One Variable at a Time and a basic feel for sampling; the formal side of intervals comes in Estimation and Confidence Intervals.

Why averages are not enough

Two groups can share the same mean and look nothing alike. The first group below is bell-shaped. The second is a mix of two kinds of customers, those who buy almost nothing and those who buy a lot. Their averages match closely, but any decision about "the typical customer" differs completely.

Python Same mean, different shapes

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The standard deviations already give a hint, but the picture is unmistakable. The rule: before you reduce a group to one number, look at its distribution.

Box plots, violins and raw points

Three plot types show distributions for several groups side by side:

  • A box plot draws the median, the quartiles (the box covers the middle 50% of values), and whiskers with outliers beyond them. It is compact and works for many groups, but it cannot show that a distribution has two peaks.
  • A violin plot draws a smoothed density, mirrored on both sides, so you see the shape, including multiple peaks.
  • A strip or swarm plot puts every observation as a dot (slightly jittered sideways). With up to a few hundred points per group it is the most honest option, and it combines nicely with a box or violin.
Python Box, violin and raw points for three groups

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The box plot shows Variant 2 with a median close to Variant 1 and only a wider box; the violin and the dots reveal that it has two clusters, with almost nobody near the median. That is the kind of finding that changes a business decision.

The ECDF: a distribution without bins

A histogram depends on the number and width of its bins, and a bad choice can hide or invent structure. The empirical cumulative distribution function (ECDF) has no such parameter. For each value on the x-axis it shows the fraction of observations at or below it. It is a staircase that rises from 0 to 1.

To build one, sort the data and plot the values against 1/n, 2/n, ..., 1. How to read it:

  • The height at x is "what share of the data is at or below x". The curve crossing 0.5 is the median.
  • A steep section means many values are packed together; a flat section means a gap.
  • If one curve is always to the right of another, that group has higher values at every percentile.
Python ECDFs make groups easy to compare

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

(For a time-to-complete measure, a curve further to the right is worse: users take longer.) The ECDF also copes with long right tails, as here, particularly with a log-scaled x-axis (ax.set_xscale("log")).

Error bars: how sure are we about the mean?

When you do show a mean, show how uncertain it is. A sample mean is an estimate; another sample would give a slightly different value. The usual way to quantify that is the standard error (the standard deviation divided by the square root of n) or a confidence interval.

The popular mistake is to draw error bars without saying what they are. Standard deviation bars describe the spread of the data; standard error bars and confidence intervals describe the precision of the mean. They have very different lengths. State which one you drew in the caption or axis note.

flowchart LR
    D["Data spread<br/>(standard deviation)"] -->|"divide by sqrt(n)"| E["Precision of the mean<br/>(standard error)"]
    E -->|"times about 2"| C["Approximate 95%<br/>confidence interval"]

Three different quantities that are often drawn as the same-looking whisker.

Python Mean with 95% intervals for several groups

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The interval for the small group is about five times as wide as the large group's, so its mean could easily be several dollars off. The large group's interval is narrow because the mean is pinned down by 400 observations. Showing intervals makes sample size visible without a lecture.

Bootstrapped intervals, drawn

The formula for a standard error works for means, but not for medians, ratios or other custom statistics. The bootstrap is a general method: resample your data with replacement many times, compute the statistic on each resample, and use the spread of those results as the uncertainty. The 2.5th and 97.5th percentiles of the bootstrap results form a 95% percentile interval.

Python Bootstrap distribution of a median

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The histogram is the picture of "how much would the median move if we sampled again". Showing it directly teaches readers more than a plus-or-minus number.

Confidence bands around a trend

For a line, the same idea gives a band: a shaded region around the fitted curve. Bootstrap the points, refit the line each time, and take the 2.5th and 97.5th percentiles of the predictions at each x. The band is narrow where data are plentiful and wide at the edges, where the fit is least certain.

Python A bootstrapped confidence band for a fitted line

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Notice that the band is narrower in the middle of the data than at either end. Note also what the band is not: it shows uncertainty about the line, not the range where new observations will fall. That range (a prediction interval) is much wider.

Honest uncertainty: some rules

  • Say what the interval is. Standard deviation, standard error, 95% confidence interval and bootstrap interval are different things; label them.
  • Show the data when you can. Raw points beside summaries stop readers from imagining a smooth, uniform group.
  • Do not hide overlap. If intervals overlap heavily, the honest message is "we cannot tell these apart yet" and a larger sample is the fix.
  • Do not read too much into overlap either. Two intervals can overlap and the difference between the groups can still be statistically clear. A direct interval for the difference answers that question properly.
  • A 95% interval is not a 95% chance that the true value is inside this particular interval. The precise meaning is covered in Estimation and Confidence Intervals.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. What can a violin plot show that a box plot cannot?

    A violin draws the smoothed density, so bimodal shapes are visible. A box plot only shows quartiles, whiskers and outliers.
  2. 2. What does the height of an ECDF curve at x = 30 mean?

    The ECDF at x is the fraction of data less than or equal to x, so it rises from 0 to 1.
  3. 3. How are standard deviation bars and standard error bars different?

    The standard error is the standard deviation divided by the square root of n, so it gets smaller with more data while the spread of the data does not.
  4. 4. What does a bootstrap do?

    Resampling with replacement mimics repeated sampling, and the spread of the statistic across resamples measures its uncertainty.
  5. 5. A confidence band around a regression line is narrowest where?

    A fitted line is best determined near the middle of the data. Toward the edges small changes in the sample move the line more.

Summary

  • A single average hides shape, spread and overlap; look at the distribution first.
  • Box plots are compact, violins show shape, and raw points show everything for small groups.
  • The ECDF needs no bins and makes comparing groups at every percentile easy.
  • Always say what your error bars are, and prefer confidence intervals or bootstrap intervals to bare standard deviation for the mean.
  • The bootstrap works for any statistic; plot its distribution or a band around a fitted line.
  • A confidence band describes the line, not where new data will fall.

Next: Communicating Results to Non-Experts, turning charts and numbers into a message people act on.