Skip to content
Tutorials Center
My learning

Topics

Lesson 24 of 37 ยท Instrumental Variables and DiD

Two-Stage Least Squares

Implement two-stage least squares from scratch with NumPy: first and second stage, correct standard errors, controls, the first-stage F-statistic and the weak instrument check.

The Wald estimator handles one binary instrument and no other variables. Real analyses have control variables, continuous treatments and more than one instrument. Two-stage least squares (2SLS) generalizes IV to all of those using two ordinary regressions.

The two stages

  1. First stage. Regress the treatment T on the instrument(s) Z and the controls X. Keep the fitted values T_hat: the part of treatment explained by the instrument and controls.
  2. Second stage. Regress the outcome Y on T_hat and the same controls X. The coefficient on T_hat is the 2SLS estimate.

T_hat contains only the variation in treatment that comes from Z (and X), so it is purged of the hidden confounder. In matrix form, with W = [T, X] and V = [Z, X], 2SLS is beta = (W' P W)^-1 W' P Y, where P projects onto the columns of V.

Do the second stage by hand only to learn. The standard errors printed by a naive second-stage regression are wrong because they use residuals built from T_hat instead of the real T. The code below computes them correctly.

Python 2SLS from scratch with correct standard errors

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

OLS is pulled upward by u. 2SLS comes back close to 1.5 with a larger, but honest, standard error.

Testing the instrument's strength

Relevance is checkable. Run the first stage and test whether the instruments jointly matter. The rule of thumb (Staiger and Stock) is a first-stage F-statistic above 10, and more recent work prefers values well above that, since 10 can still allow noticeable bias.

For one instrument, F equals the squared t-statistic of its coefficient in the first stage. For several, compare the model with and without the instruments.

Python First-stage F-statistic

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

A strong instrument gives an F in the hundreds; the weakest one falls below 10, which is a warning to look for a better instrument or use methods that are robust to weak instruments (such as Anderson-Rubin confidence intervals).

Controls, several instruments and overidentification

  • Controls. Include the same exogenous controls in both stages. They don't have to be confounders, but they can reduce noise and make the independence assumption more plausible ("as good as random given X").
  • Several instruments. More instruments than treatments is called overidentification. 2SLS uses them jointly. You can run a Sargan-Hansen J test: if the instruments imply different effect sizes, at least one is invalid. It cannot prove validity, since all of them could be invalid in the same way.
  • Several treatments. Each endogenous variable needs at least one instrument.

Using a library

In practice you would use a tested package, for example linearmodels.iv.IV2SLS, which adds robust standard errors, the F test and the J test. Knowing the matrix formulas above lets you read its output intelligently. Such packages are not part of the browser runner, so the examples here stay with NumPy.

Common mistakes

  1. Running the second stage manually and trusting its standard errors.
  2. Leaving a control out of the first stage but keeping it in the second. All exogenous variables belong in both.
  3. Plugging a logistic first stage into the second stage for a binary treatment (the "forbidden regression"). A linear first stage is the safe choice.
  4. Reporting an ATE. Remember that 2SLS describes compliers.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What are the two stages of 2SLS?

    The first stage isolates the part of T driven by the instrument; the second uses only that part.
  2. 2. Why are the standard errors from a hand-run second stage wrong?

    The residual variance must use the actual treatment: y minus W times beta.
  3. 3. A first-stage F-statistic is 4. What does that suggest?

    Below the rule-of-thumb threshold of 10, the instrument barely predicts treatment.
  4. 4. What does a Sargan-Hansen J test examine?

    With more instruments than treatments, disagreement between them signals that at least one is invalid.

Summary

  • 2SLS: regress treatment on instruments and controls, then regress the outcome on the fitted treatment and the controls.
  • Compute standard errors with the true treatment in the residuals, or use a library.
  • Check relevance with the first-stage F-statistic (well above 10).
  • Extra instruments allow an overidentification test but cannot prove validity.
  • The result is a LATE for compliers, not automatically an ATE.

Next: Difference-in-Differences