Skip to content
Tutorials Center
My learning

Topics

Lesson 9 of 57 ยท Describing Data

Quartiles, IQR and Outliers

Learn percentiles, quartiles, the five-number summary and the interquartile range, and use the 1.5 times IQR rule and box plots to detect outliers in your data.

The standard deviation is sensitive to outliers, because it squares the deviations. A resistant way to describe spread uses positions in the sorted data instead: percentiles and quartiles. This lesson builds them up, introduces the five-number summary and shows how to spot outliers.

Percentiles

The p-th percentile is a value such that about p% of the data lie at or below it. The 50th percentile is the median. If your exam score is at the 90th percentile, you did better than roughly 90% of the test takers. Percentiles are used for exam results, growth charts for children and service-level targets (a "p95 response time").

Quartiles

The quartiles split sorted data into four equal parts:

  • Q1 (25th percentile): a quarter of the data are below it.
  • Q2 (50th percentile): the median.
  • Q3 (75th percentile): three quarters are below it.

There are several conventions to compute them. A simple one: sort the data, take the median, then take the median of the lower half for Q1 and of the upper half for Q3. Software uses interpolation between positions, so results can differ a little between tools. Always say which method you use.

For 3, 5, 7, 8, 12, 13, 14, 18, 21: the median is 12; the lower half (3, 5, 7, 8) has median 6 and the upper half (13, 14, 18, 21) has median 16. So Q1 = 6, Q3 = 16.

The interquartile range

The IQR is Q3 - Q1: the spread of the middle 50% of the data. Unlike the range or the standard deviation, extreme values at either end have no effect on it. Here IQR = 16 - 6 = 10.

Python Quartiles and IQR in NumPy

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

NumPy's default (linear interpolation) gives Q1 = 7 and Q3 = 14 for this data, while the (n + 1) definition and the halves method give 6 and 16. Neither is wrong; they are different definitions. With large samples the difference vanishes.

The five-number summary

A compact description of any distribution: minimum, Q1, median, Q3, maximum. It is the basis of the box plot and tells you center, spread and skew at a glance: if the median is closer to Q1 than to Q3, the upper half is more stretched out (right skew).

Python Five-number summary with pandas

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Outliers and the 1.5 ร— IQR rule

An outlier is a value far from the rest. Tukey proposed a practical rule: compute the fences

  • lower fence = Q1 - 1.5 * IQR
  • upper fence = Q3 + 1.5 * IQR

Values outside the fences are flagged as potential outliers. Values beyond 3 * IQR are sometimes called extreme outliers.

Python Detect outliers with the fences

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The value 45 is flagged. A flagged value is a prompt to investigate, not an order to delete it. Possible causes: a typing error (450 instead of 45), a different kind of case (a patient with a rare condition), or genuine rare but valid variation. Remove data only with a documented reason.

Never drop outliers just to make results look nicer. Report analyses with and without them when the decision matters.

Box plots

A box plot (box-and-whisker plot) draws the five-number summary: a box from Q1 to Q3 with a line at the median, whiskers reaching to the most extreme values still inside the fences, and individual dots for outliers. It lets you compare several groups side by side.

flowchart LR
    A["min or lower whisker"] --- B["Q1"]
    B --- C["median"]
    C --- D["Q3"]
    D --- E["max or upper whisker"]
    F(("outlier")) -.-> E

The box holds the middle 50% of values. Whiskers and dots show the tails.

Python A box plot comparing two groups

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Other robust spread measures

  • Median absolute deviation (MAD): the median of |x - median|. Very resistant, used in anomaly detection.
  • Percentile ranges: e.g. the 10th to 90th percentile range.

The z-score rule is the usual alternative for outliers: flag values more than 3 standard deviations from the mean. It works best for bell-shaped data and is weakened by the very outliers it tries to find (they inflate the standard deviation).

Percentile rank

The percentile rank of a value is the percentage of data at or below it. If 54 of 60 students scored at most 82, then 82 has a percentile rank of 90.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Q1 = 20 and Q3 = 36. What is the IQR?

    IQR = Q3 - Q1 = 36 - 20 = 16.
  2. 2. With Q1 = 20, Q3 = 36 and IQR = 16, what is the upper outlier fence?

    36 + 1.5 * 16 = 60.
  3. 3. Which spread measure is least affected by one extreme value?

    The IQR uses only the middle 50% of the sorted data.
  4. 4. What should you do after an outlier is flagged?

    An outlier may be an error or a real, important case. Check before deciding.

Summary

  • Percentiles and quartiles describe position within sorted data; Q2 is the median.
  • IQR = Q3 - Q1 measures the spread of the middle half and is resistant to outliers.
  • The five-number summary (min, Q1, median, Q3, max) underlies the box plot.
  • The 1.5 ร— IQR fences flag potential outliers; investigate them before removing anything.
  • Different software uses different quartile definitions, so state your method.

Next: Skewness and Kurtosis