Skip to content
Tutorials Center
My learning

Topics

Lesson 24 of 36 ยท Panel and Time Series

Fixed Effects, Random Effects and the Hausman Test

Estimate panel models in Stata with xtreg, fe and re, cluster the standard errors, compare them with the Hausman test and see the within transformation work.

Panel data lets you control for everything about a unit that does not change over time, even things you cannot measure: a worker's talent, a firm's culture, a country's geography. The tool is the unit-specific intercept. Stata's xtreg command offers three estimators with different assumptions: pooled OLS, fixed effects (FE) and random effects (RE). Choosing between them is one of the most common decisions in applied work.

Stata cannot run here, so Stata commands are shown read-only. The runnable blocks are Python stand-ins that implement the within transformation, the random-effects estimator and the Hausman statistic from scratch on simulated data. They illustrate the method; the numbers are not Stata output.

The model

Write the outcome for unit i in period t as y_it = x_it * beta + a_i + e_it. The term a_i is the unit effect. If a_i is correlated with the regressors (talented people also get more education), pooled OLS and RE are biased. Fixed effects removes a_i by working only with differences from each unit's own average.

Commands

Declare the panel first (see panel data and xtset):

STATA
xtset id year
xtreg wage educ exper union, fe
estimates store fe
xtreg wage educ exper union, re
estimates store re

The FE output reports the within, between and overall R-squared, the estimated sigma_u (standard deviation of the unit effects), sigma_e and rho, the share of variance due to the unit effect. It also prints an F test that all unit effects are zero. A significant test says pooled OLS is inappropriate.

What fixed effects cannot do: any variable that does not change within a unit (sex, birth cohort, country region) is absorbed by the unit intercept and drops out with a note. That is the price of controlling for everything fixed.

An equivalent way to see FE is to add a dummy for every unit, regress wage educ exper i.id; xtreg, fe gives identical slopes without printing hundreds of dummies. For models with several high-dimensional effects, reghdfe (user-written) is a popular alternative.

Random effects

RE treats a_i as a random draw uncorrelated with the regressors, and uses quasi-demeaning: it subtracts a fraction theta of each unit's mean. It is more efficient than FE and can estimate time-invariant variables, but is consistent only if the zero-correlation assumption holds. In practice, that assumption is often doubtful.

Choosing: the Hausman test

If both estimators are consistent (RE assumption true), their coefficients should be close. If they differ systematically, the RE assumption fails and you should prefer FE.

STATA
hausman fe re

The null hypothesis is that RE is consistent and efficient; a small p-value rejects it, favouring FE. The classic hausman test requires homoskedastic, non-clustered standard errors. If you need cluster-robust inference, use the robust alternatives: xtoverid (user-written) after xtreg, re vce(cluster id), or estimate a Mundlak (correlated random effects) regression that adds the unit means of each time-varying regressor and test whether their coefficients are zero.

Stand-in: FE, RE and the Hausman statistic

The simulation makes the unit effect correlated with x on purpose. FE should recover the true slope of 1.0 and the RE and pooled estimates should be too high.

Python FE vs RE vs pooled, with Hausman (Python stand-in)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Pooled and RE estimates are pulled above the truth by the correlation between x and the unit effect; the Hausman statistic is large, so we would choose FE. Change 0.8 * a to 0.0 * a and run it again: the estimators agree and the test no longer rejects, so the efficient RE estimator would be fine.

Time effects and clustering

Add period dummies to absorb shocks common to everyone in a given year, such as a recession:

STATA
xtreg wage educ exper i.year, fe vce(cluster id)
testparm i.year

vce(cluster id) allows errors to be correlated over time within a person. It is the default recommendation for panel work because the idiosyncratic errors usually are. testparm i.year jointly tests whether the year effects matter.

The between estimator xtreg, be uses only differences across units, and xtreg, fe only differences within them; RE is a weighted combination of both.

Stand-in: the within transformation, step by step

Python What demeaning does (Python stand-in)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

After demeaning, the big differences between units (10s, 20s, 30s) are gone, so a unit's fixed characteristics cannot explain what is left. Only changes over time remain.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Which variable cannot be estimated in a fixed-effects model?

    Time-invariant variables are absorbed by the unit effects and drop out.
  2. 2. The null hypothesis of the Hausman test is that

    Rejecting the null suggests the unit effect is correlated with the regressors, so FE is preferred.
  3. 3. Which option gives cluster-robust inference in xtreg?

    vce(cluster id) allows arbitrary within-unit error correlation.
  4. 4. Fixed effects removes the unit effect by

    The within transformation demeans every variable inside each unit.

Summary

  • xtreg, fe controls for all time-invariant unit traits but cannot estimate them.
  • xtreg, re is more efficient but requires the unit effect to be uncorrelated with regressors.
  • hausman fe re compares them; use Mundlak or cluster-robust alternatives when errors are not iid.
  • Use vce(cluster id) and period dummies in most applied panel models; the Python blocks are stand-ins.

Next: Time series with tsset