Skip to content
Tutorials Center
My learning

Topics

Lesson 30 of 39 ยท Regression and ANOVA

Multiple Regression in SPSS: Several Predictors

Fit a multiple regression in SPSS: entering several predictors, dummy variables, adjusted R squared, standardized betas, VIF and tolerance, and stepwise versus enter.

Real outcomes depend on more than one thing. Salary depends on experience, education and city; an exam score depends on study hours, sleep and prior grades. Multiple regression puts several predictors in one model and gives each its own slope holding the others constant:

y = b0 + b1 x1 + b2 x2 + ... + bk xk + error

This is the workhorse behind most applied statistics in SPSS. If you have not read simple linear regression yet, start there, because the same tables appear.

Holding the others constant

The slope b1 answers: "if x1 rises by one unit while x2 and the other predictors stay the same, how much does y change?" That is why a predictor's coefficient in a multiple model can differ from its simple-regression coefficient: the other variables soak up shared variation.

Python Stand-in: a two-predictor model solved with matrices

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Running it in SPSS

  1. Analyze > Regression > Linear.
  2. Dependent: the outcome. Independent(s): all predictors (scale or dummy-coded).
  3. Method: Enter puts all predictors in at once. This is the sensible default when theory tells you the predictors.
  4. Statistics: Estimates, Confidence intervals, Model fit, R squared change (for hierarchical models), Collinearity diagnostics, Durbin-Watson, Casewise diagnostics.
  5. Plots: *ZRESID against *ZPRED, histogram and normal probability plot.
  6. Paste:
Text
REGRESSION
  /MISSING LISTWISE
  /STATISTICS COEFF OUTS CI(95) R ANOVA COLLIN TOL CHANGE
  /CRITERIA=PIN(.05) POUT(.10)
  /NOORIGIN
  /DEPENDENT score
  /METHOD=ENTER hours sleep
  /SCATTERPLOT=(*ZRESID ,*ZPRED)
  /RESIDUALS DURBIN HISTOGRAM(ZRESID) NORMPROB(ZRESID).

Reading the output

  • Model Summary: R, R Square, Adjusted R Square and Std. Error of the Estimate. R squared can only rise when you add predictors, even useless ones; adjusted R squared penalizes that. Use the adjusted value to compare models of different size.
  • ANOVA: tests all slopes together (H0: all are zero).
  • Coefficients: for each predictor B, Std. Error, Beta, t, Sig. and 95% CI. Standardized Beta lets you compare predictors measured in different units (the one with the larger absolute Beta carries more weight in the model).
  • Collinearity Statistics: Tolerance and VIF (variance inflation factor, = 1 / tolerance).

Multicollinearity

When predictors are strongly correlated with each other, the model cannot tell their effects apart: standard errors inflate, signs flip, and a variable that clearly matters can appear non-significant.

  • VIF above about 5 (some use 10) or tolerance below about .2 (.1) signals trouble.
  • Remedies: drop or combine redundant predictors (for example average two similar scales), or centre predictors when you include interactions.
Python Stand-in: computing VIF

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The x1 and x3 columns get huge VIFs, which is exactly the warning SPSS shows.

Categorical predictors need dummy variables

A nominal predictor with k categories enters as k - 1 dummy (0/1) variables; SPSS's Linear dialog has no "Categorical" button, so create them yourself with Transform > Recode into Different Variables or syntax. Omitted category = reference group. Each dummy's B is the difference between that category and the reference, controlling for the other predictors. (Binary logistic regression, in contrast, has a Categorical button; see logistic regression.)

Text
RECODE region (1=1) (ELSE=0) INTO north.
RECODE region (2=1) (ELSE=0) INTO south.
EXECUTE.

Here region 3 is the reference. The full syntax toolkit is in data steps in syntax.

Entering predictors: Enter, blocks and stepwise

  • Enter: all at once, based on theory. Preferred.
  • Blocks (hierarchical): press Next in the dialog to add a second set of predictors and request R squared change to see how much the new block adds.
  • Stepwise / Forward / Backward: SPSS lets the data pick predictors. It capitalizes on chance and gives inflated results in small samples; avoid it for confirmatory work.

Sample size

A common rule of thumb is at least 10 to 20 cases per predictor (or 50 + 8k overall). Fewer cases give unstable coefficients and a misleadingly high R squared.

Reporting

Give a table with B, standard error, Beta, p and the CI for each predictor, then one sentence such as: "The model explained 99% of the variance in score (adjusted R squared = .99); each extra hour of study added 3.5 points, holding sleep constant." (Numbers from the first stand-in above; add F, Beta and p from your own output.)

Common mistakes

  • Dropping variables solely because p > .05 in a collinear model.
  • Interpreting B for dummy variables without remembering the reference category.
  • Comparing unstandardized B across variables with different units.
  • Using stepwise on a small data set and reporting its p values as if the selection never happened.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why use adjusted R squared when comparing models?

    R squared never falls when you add a predictor; the adjusted version corrects for the number of predictors.
  2. 2. A VIF of 12 for a predictor indicates:

    High VIF (tolerance near 0) means the predictor is nearly a linear combination of the others.
  3. 3. A nominal variable with 4 categories needs how many dummy variables?

    k categories need k - 1 dummies; the omitted category is the reference.
  4. 4. Which method is generally preferred for a theory-driven model?

    Enter keeps your theory in control; automatic selection capitalizes on chance.

Summary

  • Multiple regression gives each predictor an effect holding the others constant.
  • Read adjusted R squared, the ANOVA F, B and Beta, and the VIF/Tolerance columns.
  • Watch for multicollinearity; dummy-code nominal predictors with a clear reference group.
  • Prefer Enter or blocks to stepwise methods, and keep enough cases per predictor.

Next: Binary Logistic Regression