Skip to content
Tutorials Center
My learning

Topics

Lesson 37 of 37 ยท Projects and Practice

Causal Inference Practice Problems

Twelve practice problems on matching, weighting, IV, difference-in-differences, RDD, uplift and pitfalls, with worked solutions and runnable Python to check your answers.

Reading about causal inference is not the same as using it. This page gives you twelve problems that mix calculation, design choice and interpretation. Try each problem on paper first, then compare with the solution. Runnable code at the end lets you check the numeric ones and experiment on your own.

Part 1: calculation problems

Problem 1 (DiD). A shop chain launched a loyalty app in 10 stores. Average weekly sales per store (in thousands): treated stores 80 before, 95 after; comparison stores 70 before, 76 after. What is the DiD estimate?

Problem 2 (Wald/IV). A randomized email encouraged people to start a fitness program. Among the 1,000 who got the email, 600 started and their average weight loss was 2.0 kg across the whole group. Among the 1,000 who did not get the email, 200 started and average weight loss was 0.5 kg across the group. Compute the Wald (IV) estimate of the effect of starting the program.

Problem 3 (IPW). A treated customer has propensity score 0.2 and a control customer has propensity score 0.2. What are their ATE weights? What is the ESS if only these two units existed with those weights?

Problem 4 (E-value). An observational study reports a risk ratio of 2.5. Compute the E-value, RR + sqrt(RR*(RR-1)).

Problem 5 (Uplift). A model predicts purchase probability 0.30 with a coupon and 0.35 without. The margin is 40 and the coupon costs 5 per purchase. Should this customer get a coupon?

Problem 6 (RDD). Students scoring 60 or higher get tutoring. Average final exam for students with scores 55 to 59.9 is 62, and for 60 to 64.9 is 68, while the fitted line slopes 0.5 points per score point on both sides. What is a rough estimate of the jump at 60?

Part 2: design and interpretation

Problem 7. A hospital finds that patients who got a new treatment have worse outcomes than those who did not. List two reasons this could happen even if the treatment helps.

Problem 8. You want the effect of a training course on salary. You have the course participation, salary, education, and "job satisfaction measured after the course". Which variables should you adjust for?

Problem 9. A company rolls out a feature in one country only and compares revenue before and after with a neighboring country as the control. Name two threats to parallel trends.

Problem 10. A study of a draft lottery uses lottery number as an instrument for military service and earnings as the outcome. What does the exclusion restriction mean here, and name one way it could fail.

Problem 11. The 20% of customers with the worst satisfaction scores received a call from the support team, and a month later their satisfaction rose. Why is this not evidence the call works?

Problem 12. An uplift model's decile table shows observed uplift 0.10, 0.08, 0.09, 0.07, 0.06, 0.05, 0.05, 0.04, 0.03, 0.02 from top to bottom decile. Is this a well-calibrated uplift model for targeting? What would you want to see in the top deciles?

Check your answers with code

Python Numeric solutions to problems 1 to 6

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Worked reasoning for the numeric answers:

  • P1: treated change +15, comparison change +6, so DiD = 9 (thousand per store per week). Assumes the treated stores would have grown like the comparison stores.
  • P2: reduced form 1.5 kg, first stage 0.4, so the effect of starting is 3.75 kg for compliers (people whose behavior was changed by the email).
  • P3: both are 1/0.2 = 5 and 1/0.8 = 1.25. ESS = 6.25^2 / (25 + 1.5625) = 1.47: two units but little information, because one dominates.
  • P4: 2.5 + sqrt(2.5 * 1.5) = 2.5 + 1.94 = 4.44: a hidden confounder would need a risk ratio of 4.4 with both treatment and outcome to explain the result away.
  • P5: with coupon 0.30 * 35 = 10.5; without 0.35 * 40 = 14. The coupon lowers expected profit (a "sleeping dog": predicted uplift is negative). Do not send it.
  • P6: bin centers are about 57.5 and 62.5, 5 points apart, and the line rises 2.5 over those 5 points, so the jump is roughly 6 - 2.5 = 3.5 points. A real analysis would use local linear regression on each side.

Part 2 solutions

  • P7: Confounding by indication (sicker patients get the new treatment) and selection (treatment given in hospitals with worse case mix) or immortal-time effects. Fix with randomization, adjustment for severity, or an instrument (for example, physician preference).
  • P8: Adjust for education (a pre-course confounder) and other pre-course variables such as prior salary. Do not adjust for post-course job satisfaction: it is a mediator or collider and would bias the effect.
  • P9: The two countries may differ in trends (different growth, seasonality, marketing), a concurrent event hit one country only (holiday, competitor launch, currency change), or the feature spilled over to the control country.
  • P10: The lottery number affects earnings only through military service. It could fail if a low (draft-eligible) number leads men to stay in school longer to get deferments, which affects earnings without service.
  • P11: Regression to the mean: extreme scores are partly noise and move toward the average on their own. You need a randomized or equally selected control group that did not get the call.
  • P12: The model ranks poorly for targeting: observed uplift shrinks steadily with rank, which is good, but the separation is small (0.10 vs 0.02) and every decile is positive, so it does not discover anyone with negative or zero uplift. You want a clear gap between top and bottom deciles, ideally with the bottom ones near zero or negative, plus a cumulative gain curve clearly above the random-targeting line on held-out randomized data.

Part 3: a simulation challenge

Try a full workflow on your own. The code below creates confounded data and hands you an unfinished analysis. Fill in the TODOs, then compare with the truth (the effect is 1.5).

Python Challenge: recover the effect from confounded data

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

If you finish quickly, break something on purpose: drop health from the propensity model and watch the estimate drift, or add a post-treatment variable and see what it does. Understanding how a method fails is the best preparation for using it on real data.

What to study next

You now have the core toolkit: potential outcomes, graphs, adjustment, matching and weighting, IV, DiD, RDD, synthetic control, heterogeneous effects, doubly robust methods and sensitivity analysis. To go deeper, read a textbook such as Causal Inference: What If (Hernan and Robins) or Causal Inference: The Mixtape (Cunningham), try libraries such as DoWhy, EconML, CausalML and linearmodels on your own machine, and, most importantly, practice on real questions from your own field, where the hard part is the argument for why the assumptions hold.

Check your understanding

Quick check ยท 3 questions

of 3 answered You scored / 3 ยท perfect!

  1. 1. In Problem 2, what does the Wald estimate of 3.75 kg describe?

    IV estimates the local average treatment effect for compliers.
  2. 2. Why does Problem 8 tell you not to adjust for job satisfaction measured after the course?

    Post-treatment variables can remove part of the effect or open spurious paths.
  3. 3. What is the best first defense against regression to the mean (Problem 11)?

    The control group shows how much improvement happens without the intervention.

Summary

  • Practice with small numbers (DiD, Wald, weights, E-values) until the formulas are automatic.
  • For design questions, ask: what is the comparison, what could make the groups differ, and which variables are before or after treatment?
  • Always try to break your own analysis with placebo, sensitivity and balance checks.
  • Keep building on real data from your own field.

Next: Return to the Causal Inference track page