Skip to content
Tutorials Center
My learning

Topics

Lesson 30 of 37 · Correlation and Regression

Residuals and R-Squared

Learn how to compute residuals, read a residual plot to check a linear model, understand R-squared, and recognize influential points.

A regression line is a model, and every model makes errors. This lesson shows how to measure those errors, how to tell whether a straight line was a sensible choice, and how to say how much of the variation your model explains.

It continues from the least-squares line.

Residuals

A residual is the difference between what actually happened and what the line predicted:

Text
residual = y - ŷ

Positive residual: the point is above the line (the model under-predicted). Negative: below it.

For the study data with ŷ = 46.3 + 4.9x:

x y ŷ Residual
1 52 51.2 0.8
2 55 56.1 -1.1
3 61 61.0 0.0
4 66 65.9 0.1
5 71 70.8 0.2

The residuals add to zero (within rounding), which is always true for a least-squares line with an intercept.

Python Compute residuals and their sum of squares

Press Run (or Ctrl + Enter) to see the result here.

Program input · what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Residual plots

Plot the residuals against x (or against ŷ). If a straight line is appropriate, the plot should look like a random cloud around zero, with no pattern and roughly constant spread.

flowchart TD
  A["Plot residuals vs x"] --> B{"Pattern?"}
  B -->|"Random scatter around 0"| C["Linear model is reasonable"]
  B -->|"Curve (U or arch)"| D["Relationship is not linear"]
  B -->|"Funnel shape"| E["Spread changes with x"]

A residual plot is a quick diagnostic for the fit.

Python A good fit and a bad fit, side by side

Press Run (or Ctrl + Enter) to see the result here.

Program input · what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The first panel looks like random noise; the second shows a clear arch, a sign that a straight line is the wrong model.

R-squared

How good is the line overall? Compare the variation left over with the variation you started with:

Text
SST = Σ(y - ȳ)²          total variation in y
SSE = Σ(y - ŷ)²          variation left in the residuals
R² = 1 - SSE / SST

For the study data, SST = 242 and SSE = 1.90 (0.64 + 1.21 + 0 + 0.01 + 0.04). So:

Text
R² = 1 - 1.90 / 242 ≈ 0.992

About 99.2% of the variation in test scores is accounted for by the linear relationship with hours studied. For a simple line, R² = r², matching the r² = 2401/2420 found earlier.

Python R-squared two ways

Press Run (or Ctrl + Enter) to see the result here.

Program input · what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

A high R² does not prove the model is right. A curved pattern can still give a high R², so always check the residual plot. And R² says nothing about causation.

Outliers and influential points

A point with an unusual y given its x has a large residual. A point with an unusual x (far from x̄) has high leverage and can pull the line toward it. A point that changes the slope noticeably when removed is influential. Try fitting with and without it; if the result changes a lot, report both.

Check your understanding

Quick check · 4 questions

of 4 answered You scored / 4 · perfect!

  1. 1. The actual score is 70 and the model predicts 65. What is the residual?

    Residual = actual - predicted = 70 - 65 = +5.
  2. 2. A residual plot shows a clear U-shaped pattern. What does that suggest?

    Systematic structure in the residuals means the linear model is missing something.
  3. 3. R² = 0.81 for a simple linear regression. What is |r|?

    For simple linear regression R² = r², so |r| = √0.81 = 0.9.
  4. 4. Which is true for a least-squares line with an intercept?

    The positive and negative residuals balance exactly.

Summary

  • Residual = actual - predicted. They sum to zero for a least-squares line.
  • A residual plot should show random scatter; patterns mean the model is wrong.
  • R² = 1 - SSE/SST is the proportion of variation explained; R² = r² for a simple line.
  • Watch for outliers and influential points.

Next: Populations, Samples and Sampling Methods