Skip to content
Tutorials Center
My learning

Topics

Lesson 35 of 36 ยท Projects and Practice

Project: Policy Evaluation with Difference-in-Differences

Evaluate a policy with a panel in Stata: check parallel trends, run a two-way fixed-effects difference-in-differences with xtreg or reghdfe, cluster errors and plot an event study.

Did a minimum-wage increase reduce employment? Did a new training programme raise earnings? Questions about policies are about causal effects, and cross-sectional regressions rarely answer them. A common design uses panel data: compare how an outcome changed over time in places that adopted the policy with how it changed in places that did not. The comparison of two changes is the difference-in-differences (DiD) estimator. In this project you will build it step by step.

Stata cannot run on this site, so Stata code is read-only, and we show no Stata output. The runnable parts are Python stand-ins on simulated data (no real policy or place is involved) that let you see the estimator and the event study work.

The idea in a table

Suppose average employment was 100 before and 96 after in treated states, and 100 before and 99 after in control states. The control group's change (-1) shows what would have happened without the policy (the common trend). The DiD estimate is (96 - 100) - (99 - 100) = -3. The key assumption is parallel trends: without the policy, the treated and control groups would have moved the same way.

Data and setup

You need a panel with a unit id, a time variable, an outcome, a treatment indicator and the policy start date.

STATA
use "data/clean/state_panel.dta", clear
xtset state year
xtdescribe
generate treated = (adopter == 1)
generate post    = (year >= 2015)
generate policy  = treated * post     // 1 only for treated units after adoption

Run xtdescribe and xtsum first (panel basics): if some states drop out, think about why.

The canonical 2x2 regression

With one adoption date:

STATA
regress emp i.treated##i.post, vce(cluster state)

The coefficient on 1.treated#1.post is the DiD estimate. Cluster at the level where the policy varies (state); with fewer than 30-40 clusters consider the wild-cluster bootstrap (boottest, user-written).

Two-way fixed effects

With many units and years, include unit and year effects instead:

STATA
xtreg emp policy i.year, fe vce(cluster state)
* or, with high-dimensional effects:
reghdfe emp policy, absorb(state year) vce(cluster state)

The coefficient on policy is the average effect, provided all units adopt at the same time. When adoption is staggered (different states adopt in different years) and effects change over time, the simple two-way FE coefficient can be biased because already-treated units serve as controls. Modern alternatives are csdid (Callaway and Sant'Anna), did_multiplegt, jwdid and eventstudyinteract (all user-written); Stata 18 added the official didregress and xtdidregress commands for the standard design.

STATA
didregress (emp) (policy), group(state) time(year)
estat trendplots
estat ptrends

Plot effects by year relative to adoption. Leads (before the policy) should be near zero; lags show the dynamic effect.

STATA
generate rel = year - adopt_year if treated
replace rel = -99 if !treated
forvalues k = 4(-1)2 {
    generate lead`k' = (rel == -`k')
}
generate lag0 = (rel == 0)
forvalues k = 1/4 {
    generate lag`k' = (rel == `k')
}
reghdfe emp lead4 lead3 lead2 lag0 lag1 lag2 lag3 lag4, absorb(state year) vce(cluster state)
coefplot, vertical keep(lead* lag*) yline(0)

Period -1 is the omitted reference. If the lead coefficients are large or trending, parallel trends is doubtful and the DiD estimate should not be trusted as causal.

Rehearsal: simulated 2x2 and two-way FE

The block makes 40 units over 10 years; units 21-40 adopt the policy in year 6 with a true effect of -3. Unit and year effects are random. It computes the 2x2 estimate and a two-way FE estimate (by demeaning in both dimensions on a balanced panel). This is a Python stand-in on simulated data.

Python DiD and two-way fixed effects (Python stand-in)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Both estimates are close to the true -3, and the standard error would be larger if we clustered by unit. The point of the double demeaning is that it removes every unit-specific and year-specific constant, just as i.year plus , fe does in Stata.

Rehearsal: event study

Python Event-study coefficients (Python stand-in)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Before adoption (negative event times) the estimates hover around zero, which supports the parallel-trends assumption. After adoption they fall, tracing the dynamic effect.

Writing it up

  1. State the design and the identifying assumption in plain words.
  2. Show the event-study graph and discuss pre-trends.
  3. Report the main estimate with clustered standard errors and the number of clusters (tables).
  4. Run robustness checks: different control groups, placebo adoption dates, dropping one unit at a time, an alternative estimator for staggered adoption.
  5. Say what the estimate does not tell you (spillovers, long-run effects, other policies changing at the same time).

For the fixed-effects background see fixed and random effects.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. The DiD estimate is

    It compares two changes over time, which removes fixed differences between groups and common shocks.
  2. 2. Which assumption is essential for DiD?

    Without parallel trends the control group's change is not a valid counterfactual.
  3. 3. At what level should you cluster standard errors for a state-level policy?

    Cluster where treatment varies, since errors are correlated within states over time.
  4. 4. Why is two-way FE problematic with staggered adoption and changing effects?

    Heterogeneous and dynamic effects make the weights in two-way FE potentially negative; use modern estimators such as csdid.

Summary

  • DiD compares changes over time between treated and control units; parallel trends is the key assumption.
  • Use regress i.treated##i.post, xtreg, fe with year dummies, reghdfe, or didregress; cluster at the policy level.
  • An event study tests pre-trends and shows dynamics; staggered adoption needs modern estimators.
  • The Python blocks are stand-ins on simulated data.

Next: Practice: Stata exercises with solutions