Skip to content
Tutorials Center
My learning

Topics

Lesson 33 of 37 ยท Practice and Pitfalls

Sensitivity Analysis for Unmeasured Confounding

Ask how strong an unmeasured confounder would have to be to explain away your result: omitted variable bias, bias tables, tipping points and the E-value, with runnable Python.

Every observational causal estimate rests on an assumption you cannot test: that you measured all the important confounders. Pretending it is certain is dishonest; pretending the study is worthless is lazy. Sensitivity analysis is the middle path. It asks: how wrong would my assumption have to be to change my conclusion?

Instead of "there is no unmeasured confounding", you can then write: "an unmeasured confounder would need to be at least as strong as X to explain this effect away". Readers (and you) can judge whether such a confounder is plausible.

Omitted variable bias, the simplest version

Suppose the true model is Y = b*T + g*U + e and U is related to treatment through T = d*U + noise. If you leave out U, the regression coefficient on T is biased by approximately g * d', where d' is the regression coefficient of U on T. The bias is the product of two things: how strongly the confounder affects the outcome (g) and how unbalanced it is across treatment groups (d'). If either is zero, there is no bias.

Python Bias from a confounder of growing strength

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

When either strength is zero the bias vanishes. The table is a miniature "bias table": a map from assumed confounder strengths to the size of the bias.

Tipping-point analysis

A tipping point is the confounder strength at which your conclusion flips (the effect becomes zero or crosses a decision threshold). Procedure: pick an effect estimate, define a family of hypothetical confounders (for example, binary U with prevalence p1 in treated and p0 in controls, and effect g on the outcome), subtract the implied bias g * (p1 - p0) from the estimate, and find where it hits zero.

Python Tipping point with a binary confounder

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

You then judge: is a confounder that raises the outcome by 8 units, and that is twice as common among treated units, believable given everything known about the field? The comparison with observed covariates helps: if your strongest measured confounder has an effect of 2, a hidden one with effect 8 would be unusually strong.

The E-value

The E-value (VanderWeele and Ding, 2017) packages this into one number for risk ratios. It is the minimum strength of association, on the risk ratio scale, that an unmeasured confounder would need with both the treatment and the outcome, beyond the measured covariates, to fully explain away the observed association.

For an observed risk ratio RR greater than 1:

E-value = RR + sqrt(RR * (RR - 1))

For RR < 1, first invert it (1/RR). Compute the same number for the confidence limit closest to 1 to see how strongly the interval could be explained away. A large E-value means a hidden confounder would have to be implausibly strong.

Python Computing E-values

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

For an RR of 3.9 the E-value is about 7.3: a confounder would need a risk ratio of 7.3 with both treatment and outcome to explain it away. For RR = 1.2 the E-value is about 1.7, a modest confounder could do it.

Practical advice

  1. Always report some form of sensitivity analysis for observational estimates.
  2. Benchmark against measured covariates. "A hidden confounder as strong as smoking would be needed" is meaningful; "an E-value of 2.1" alone is not.
  3. Use negative controls. An outcome that the treatment cannot cause, or a treatment that cannot cause the outcome, should show no effect. If it does, your adjustment misses confounding.
  4. Consider bounds methods. Rosenbaum bounds (for matched studies) and Cinelli and Hazlett's partial R-squared approach for regression give richer versions of the same idea.
  5. Remember the limits. Sensitivity analysis quantifies the risk; it never removes it.
flowchart LR
    A["Estimate"] --> B["Assume a hidden confounder of strength s"]
    B --> C["Compute adjusted estimate"]
    C --> D{"Conclusion still holds?"}
    D -->|"Yes for plausible s"| E["Robust result"]
    D -->|"No"| F["Fragile result"]

Vary the assumed strength until the conclusion flips, then judge plausibility.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does omitted variable bias depend on?

    The bias is roughly the product of the two: no effect on the outcome or no imbalance means no bias.
  2. 2. What does a large E-value tell you?

    The E-value is the minimum risk-ratio strength a hidden confounder needs with both treatment and outcome.
  3. 3. What is a negative control outcome?

    Finding an effect where none can exist reveals bias in the design.
  4. 4. Sensitivity analysis can:

    It describes fragility; it does not remove the problem.

Summary

  • Unmeasured confounding cannot be tested away; sensitivity analysis measures how much it would take to overturn a result.
  • Bias is driven by a confounder's link to both treatment and outcome.
  • Tipping points and E-values express the needed strength in one number; benchmark against measured covariates.
  • Negative controls and bounds methods complement them.

Next: Common Causal Pitfalls