Skip to content
Tutorials Center
My learning

Topics

Lesson 17 of 42 ยท Statistical and Scientific Charts

Trendlines and Error Bars

Add statistics to Plotly charts: OLS, LOWESS, rolling and expanding trendlines in Express, read the fit results, draw error bars from data, percent or constants, and shade confidence bands.

A scatter plot shows data. A trendline summarizes it, and error bars show how uncertain it is. Together they turn a picture into evidence: "tips grow with the bill by about 11 cents per dollar, give or take this much".

Plotly Express can fit trendlines for you with one argument. For anything more specific, you compute the statistic with pandas, NumPy or statsmodels and draw it as an extra trace. This lesson does both.

Trendlines in Express

Add trendline="ols" to px.scatter and Express fits a straight line by ordinary least squares (using the statsmodels package) and draws it as an extra trace for each group:

Python OLS trendline and its fit results

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The slope tells you that each extra dollar on the bill goes with about 10 cents more tip, and R-squared (about 0.46) says the bill explains about 46% of the variation in tips. px.get_trendline_results(fig) gives you the full statsmodels model, with every statistic the library knows about it. Reading these numbers instead of only looking at the line is what makes the chart honest.

With color="smoker", Express fits one line per group. To fit one line to everything while keeping the colored points, use trendline_scope="overall".

trendline= What it draws
"ols" Straight least-squares line
"lowess" Smooth local fit that follows curves (trendline_options=dict(frac=0.5) sets smoothness)
"rolling" Rolling mean (needs trendline_options=dict(window=5))
"expanding" Cumulative mean up to each point
"ewm" Exponentially weighted mean

For OLS, trendline_options=dict(log_x=True) fits a power law (a straight line in log-log space), and add_constant=False forces the line through the origin.

Python Compare trendline types

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The data follow a logarithmic curve. The straight OLS line fits poorly in the middle (positive residuals there, negative at the ends), while LOWESS follows the curve. If a pattern remains in the residuals, a straight line is the wrong model. The rolling line has fewer points because the first window - 1 rolling values do not exist.

Error bars

Error bars show uncertainty: a measurement error, a standard deviation, a confidence interval. Each point gets a vertical (error_y) or horizontal (error_x) whisker.

In Express, give a column of error sizes:

Python
px.scatter(df, x="dose", y="response", error_y="sd")
px.bar(df, x="group", y="mean", error_y="ci_up", error_y_minus="ci_down")   # asymmetric

In graph objects, error_y is a dictionary describing how the error is computed:

type Meaning
"data" Use the values from array (and optionally arrayminus for asymmetric bars)
"constant" The same value for every point
"percent" A percentage of each point's value
"sqrt" The square root of each value, handy for count data
Error bars and a confidence band

The shaded band is a plain trick: draw the upper boundary as a line with no width, draw the lower boundary with fill="tonexty" (fill up to the previous trace) and a semi-transparent color. Error bars and bands show the same information in different ways: bars are precise at each point, bands look cleaner for dense series.

A computed confidence interval

Error bars are only as good as the statistic behind them. Compute the mean and a 95% interval per group with pandas and draw them:

Python Mean with a 95% confidence interval

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Friday has the fewest orders (19), so its interval is widest. This is the main message of error bars: uncertainty depends on the sample size, and a mean without an interval can be misleading. State in the title or caption what the bars represent (standard deviation, standard error or confidence interval), because they are very different sizes.

Common mistakes

  • Showing a trendline without looking at the residuals or R-squared.
  • Extrapolating a trendline far outside the data range.
  • Leaving error bars unexplained: "ยฑ what?" is the first question a reader asks.
  • Using standard deviation bars when you mean to compare means (use standard error or a confidence interval).
  • Fitting a straight line to curved data; try LOWESS or a log transform first.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Which package does trendline="ols" need?

    Express uses statsmodels for OLS and LOWESS trendlines.
  2. 2. How do you get the fitted slope and R-squared from an Express figure with a trendline?

    get_trendline_results returns a DataFrame that holds the fitted statsmodels model for each trendline.
  3. 3. Which error_y type draws bars whose size is a fixed percentage of each value?

    percent scales the error with the value; constant uses the same size for all points.
  4. 4. How do you draw a shaded band between two lines?

    tonexty fills the area between the trace and the one drawn before it.
  5. 5. A trendline's residuals show a clear curved pattern. What does that suggest?

    Residuals should look like random noise; a pattern means structure the model misses.

Summary

  • trendline="ols" | "lowess" | "rolling" | "expanding" | "ewm" adds a fit; px.get_trendline_results returns the statsmodels model.
  • Check R-squared and residuals, and avoid extrapolation.
  • Error bars come from error_x / error_y (data, constant, percent, sqrt) and can be asymmetric.
  • Compute intervals with pandas and SciPy; say what the bars mean; use fill="tonexty" for bands.

Next: Scatter Matrix and Parallel Coordinates