Skip to content
Tutorials Center
My learning

Topics

Lesson 32 of 37 ยท Practice and Pitfalls

Doubly Robust Estimation

Combine an outcome model and a propensity model with the AIPW estimator: it stays consistent if either model is right. Includes cross-fitting and influence-function standard errors.

You now have two ways to adjust for measured confounding. Outcome regression models E[Y | X, T] and predicts what each unit's outcome would be under treatment and under control. Inverse probability weighting models the treatment assignment and reweights. Each can fail: the regression if the outcome model is wrong, the weighting if the propensity model is wrong.

Doubly robust (DR) estimators combine both so that the estimate is consistent if at least one of the two models is correct. You get two chances to be right. The best known is the augmented inverse probability weighting (AIPW) estimator.

The AIPW formula

Let mu1(x) and mu0(x) be outcome-model predictions under treatment and control, and e(x) the propensity score. For each unit compute

psi = mu1(x) - mu0(x) + T*(Y - mu1(x))/e(x) - (1-T)*(Y - mu0(x))/(1-e(x))

and average it: ATE = mean(psi).

Read it in two parts. The first part mu1 - mu0 is the regression estimate. The second part is an inverse-probability-weighted average of the residuals, a correction for outcome-model errors. If the outcome model is perfect, the residuals average to zero and the correction does no harm. If the outcome model is wrong but the propensity model is right, the weighted residuals repair the bias.

Three scenarios

We simulate data and then deliberately break one or both models, and compare three estimators. To break the outcome model we make the true outcome curved in x2 but fit a straight line; to break the propensity model we make the true assignment depend on x2 squared but fit a logistic regression on the plain covariates.

Python Outcome model, IPW and AIPW when one model is misspecified

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

When one model is wrong, AIPW stays near 2 while the estimator that depends on the broken model drifts away. When both are wrong, nothing saves you. Doubly robust is not "model-free"; it is insurance against a single failure.

"Right" here is mathematical: the model must be correctly specified, or flexible enough to capture the truth. In practice, use flexible models for both parts and check balance and overlap.

Machine learning and cross-fitting

Flexible learners (random forests, gradient boosting, lasso) reduce misspecification. But fitting them on the same data they predict can overfit and bias the estimate. Cross-fitting fixes this: split the data into folds, fit the nuisance models (mu, e) on the other folds, and predict for each fold's own units. Each unit's psi uses predictions from models that never saw it. With cross-fitting, nuisance models only need to converge at a slow rate (faster than n^-1/4), which is why this idea underlies double machine learning and TMLE.

The values psi also give you the standard error for free: SE = std(psi) / sqrt(n).

Python Cross-fitted AIPW with gradient boosting and a standard error

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

An interval that covers 2.0 is the goal. Note the clipping of the propensity score at [0.02, 0.98]: it protects against extreme weights, the same overlap concern as in plain IPW.

When to use doubly robust methods

  • Default for observational ATE/ATT estimation when you have decent covariates and enough data.
  • Randomized trials also benefit: an outcome model on baseline covariates reduces variance (covariate adjustment) while randomization guarantees the propensity part is right.
  • Remember what DR does not do: it does not fix unmeasured confounding, violations of overlap or measurement error. Pair it with sensitivity analysis.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does doubly robust mean for AIPW?

    Either model being right is enough; both being wrong still gives bias.
  2. 2. In the AIPW formula, what does the weighted-residual term do?

    It averages weighted outcome-model residuals, which are zero if the model is perfect and repair bias otherwise.
  3. 3. Why use cross-fitting with machine learning nuisance models?

    Predictions on the training data are too optimistic; cross-fitting uses held-out folds.
  4. 4. Which problem does a doubly robust estimator NOT solve?

    DR protects against model misspecification of measured variables, not against missing confounders.

Summary

  • AIPW combines an outcome model and a propensity model; it is consistent if either is correct.
  • The per-unit values give a ready standard error.
  • Cross-fit flexible machine learning nuisance models to avoid overfitting bias.
  • Clip extreme propensities; DR does not rescue poor overlap or unmeasured confounding.

Next: Sensitivity Analysis