Skip to content
Tutorials Center
My learning

Topics

Lesson 20 of 28 ยท Other Models

Quantile Regression

Model the median and other quantiles with QuantReg in Statsmodels. See why OLS misses changing spread, fit several quantile lines and use them as prediction bands.

OLS describes the average outcome for given predictors. But averages hide a lot. Does a new drug help the worst-off patients more than the typical one? How does the cost of a delayed delivery behave at the extreme 95th percentile, which is what service guarantees care about? When the spread of the data changes with the predictors, one average line tells an incomplete story.

Quantile regression fits a line for any chosen quantile: the 0.5 quantile is the median (a robust alternative to the mean), 0.9 is the level that 90% of outcomes fall below. Instead of squared errors, it minimizes an asymmetric absolute loss (the "pinball" loss), so it needs no normality assumption and is naturally resistant to outliers in y.

Fit a median line

Statsmodels offers smf.quantreg(formula, data).fit(q=0.5).

Python Median regression versus OLS with outliers

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The median line barely notices the outliers, while the OLS intercept is pulled up. This robustness is one reason to prefer it for skewed outcomes such as incomes and prices.

Several quantiles at once

The real power appears when you fit many quantiles. If the lines fan out, the spread grows with x (heteroscedasticity). Instead of "fixing" that problem as OLS does, quantile regression describes it.

Python Fit the 10th, 50th and 90th percentiles

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The slope increases with q: the 90th percentile climbs faster than the 10th. A single OLS slope would give only the middle ground. For this data the truth is a slope of 2 + 0.6 times the normal quantile (about 1.2 at q=0.1, 2.0 at 0.5 and 2.8 at 0.9).

Using quantile lines as prediction bands

The 10th and 90th percentile lines form an 80% prediction band that adapts to the changing spread: wide where the data are noisy, narrow where they are tight. Check calibration by counting how many points lie below each line: about 10% below the 0.1 line and 90% below the 0.9 line.

Python Check the coverage of the band

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Practical notes

  • The summary shows a pseudo R-squared specific to the chosen quantile; it is not comparable to OLS R-squared.
  • Quantile lines for different q can cross in sparse regions. That is a sign of limited data or misspecification; check with a plot.
  • Standard errors rely on asymptotic assumptions; with small samples consider bootstrapping.
  • The formula interface supports everything from Formulas and Patsy: transformations, categories and interactions.
  • For many quantiles, loop over q and collect the coefficients into a table to see how each effect changes across the distribution.

Quantile regression and robust regression (RLM, see Weighted and Robust Regression) both resist outliers, but they answer different questions: RLM estimates the typical mean without outlier influence, quantile regression estimates any chosen part of the distribution.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does QuantReg with q=0.5 estimate?

    q=0.5 gives median regression, a robust alternative to the mean.
  2. 2. Quantile lines fan out as x increases. What does that show?

    Different slopes at different quantiles mean the effect of x differs across the distribution, i.e. changing spread.
  3. 3. Which loss does quantile regression minimize?

    The check or pinball loss weighs over- and under-predictions differently according to q.
  4. 4. How can the 0.1 and 0.9 quantile lines be used together?

    About 80% of observations lie between the 10th and 90th percentile lines.

Summary

  • Quantile regression models any quantile of y given x, with smf.quantreg(...).fit(q=...).
  • The median line is robust to outliers; several quantiles reveal changing spread.
  • The 10th-90th lines make adaptive prediction bands; check their coverage.
  • Quantile pseudo R-squared is not comparable with OLS R-squared.

Next: LOWESS and KDE