Skip to content
Tutorials Center
My learning

Topics

Lesson 29 of 39 ยท Regression and ANOVA

Simple Linear Regression in SPSS

Predict one variable from another in SPSS: the Linear Regression dialog, the Model Summary, ANOVA and Coefficients tables, residual plots and how to write up the equation.

Correlation says two variables are related. Linear regression goes further: it fits a straight line so you can predict one variable from another and say how much the outcome changes per unit of the predictor. "Each extra hour of study is associated with about 3 more points on the exam."

The model

y = b0 + b1 * x + error

  • y: the dependent (outcome) variable; x: the independent (predictor) variable.
  • b0 (the constant, or intercept): the predicted y when x = 0.
  • b1 (the slope): the predicted change in y for a one-unit rise in x.
  • The line is chosen by least squares: it minimizes the sum of squared vertical distances (the residuals) between the points and the line.
Python Stand-in: least squares by hand

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Running it in SPSS

  1. Analyze > Regression > Linear.
  2. Dependent: score. Independent(s): hours.
  3. Statistics: keep Estimates and Model fit; add Confidence intervals (95%) and Descriptives if you wish.
  4. Plots: put *ZRESID on Y and *ZPRED on X, and tick Histogram and Normal probability plot of the standardized residuals.
  5. Save (optional): predicted values and residuals become new variables in the data.
  6. Paste:
Text
REGRESSION
  /MISSING LISTWISE
  /STATISTICS COEFF OUTS CI(95) R ANOVA
  /CRITERIA=PIN(.05) POUT(.10)
  /NOORIGIN
  /DEPENDENT score
  /METHOD=ENTER hours
  /SCATTERPLOT=(*ZRESID ,*ZPRED)
  /RESIDUALS HISTOGRAM(ZRESID) NORMPROB(ZRESID).

Reading the four tables

  1. Variables Entered/Removed: which predictors went in and the method (Enter).
  2. Model Summary: R (the correlation between observed and predicted y; for one predictor, the absolute correlation), R Square (share of variance in y explained), Adjusted R Square (corrected for the number of predictors) and Std. Error of the Estimate (the typical size of a prediction error, in y's units).
  3. ANOVA: tests whether the model as a whole predicts better than just using the mean. For simple regression, its p equals the slope's p.
  4. Coefficients: one row per term.
Column Meaning
B (Unstandardized) Intercept in the (Constant) row, slope in the predictor's row
Std. Error Precision of B
Beta (Standardized) Slope in standard deviation units; equals r in simple regression
t and Sig. Test of whether B differs from 0
95% CI for B Plausible range for the true slope

Write the fitted equation from the B column: score = 43.87 + 3.84 * hours (the fit from the first stand-in above).

Prediction versus extrapolation

You can predict within the range of your data (1 to 10 hours). Predicting for 40 hours of study assumes the line continues, which it almost certainly does not. Report predictions only inside the observed range.

Checking the assumptions

Regression has the most assumptions of anything so far, and all of them concern the residuals or the design:

  • Linearity: in the *ZRESID vs *ZPRED plot, points should scatter randomly around 0, without a curve.
  • Homoscedasticity (constant spread): no funnel shape in that plot.
  • Normality of residuals: histogram and P-P plot roughly normal (matters most for small samples).
  • Independence: not time-ordered or clustered; Durbin-Watson (in Statistics) checks time-ordered residuals, values near 2 are fine.
  • Outliers and influence: standardized residuals beyond +/-3, and Cook's distance (from Save) above about 1, deserve attention. Check the "Casewise Diagnostics" table.
Python Stand-in: residual diagnostics

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Reporting

"Hours studied significantly predicted exam score, R squared = .99. Each extra hour was associated with 3.8 more points (b = 3.84, 95% CI [3.48, 4.20])." Add the F test from the ANOVA table, and substitute values from your own output. A scatterplot with the fitted line (Chart Builder > Scatter, then add a fit line in the Chart Editor, see Chart Builder) makes the result visible.

Common mistakes

  • Reading R Square as accuracy of each prediction; the standard error of the estimate says that.
  • Claiming causation from an observational slope.
  • Fitting a line to a curve (check the residual plot).
  • Using a predictor with almost no variation.

When more than one predictor matters, move on to multiple regression.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. In the Coefficients table, which column gives the slope in the original units?

    B in the predictor's row is the change in y per one-unit increase in x. Beta is in standard deviation units.
  2. 2. R Square = .36. What does it mean?

    R squared is the proportion of outcome variance accounted for by the model.
  3. 3. What should the plot of *ZRESID against *ZPRED look like if the assumptions hold?

    A random band signals linearity and constant variance. Curves and funnels signal problems.
  4. 4. Why avoid predicting far outside the data range?

    Extrapolation assumes the pattern continues, which you have no evidence for.

Summary

  • Linear regression fits y = b0 + b1 x by least squares; Analyze > Regression > Linear.
  • Read Model Summary (R Square), ANOVA (overall fit) and Coefficients (B, CI, Sig.).
  • Check residual plots for linearity, equal spread and normality; look for outliers.
  • Predict only within the data range and avoid causal language.

Next: Multiple Regression