Skip to content
Tutorials Center
My learning

Topics

Lesson 3 of 37 ยท Getting Started

Causal Questions and Estimands

Learn to turn a vague business or research question into a precise causal estimand: treatment, outcome, population, comparison and time frame, with simple Python examples.

Before you pick a method you must know exactly what number you want. Many failed analyses answer a slightly different question than the one stakeholders cared about. A well-posed causal question has five parts, and the number that answers it is called the estimand.

The five parts of a causal question

  1. Treatment (or exposure, intervention): the thing you could change, such as "offer a 10% coupon".
  2. Outcome: what you measure afterwards, such as "purchases in the next 30 days".
  3. Population: who the answer is for, such as "all active customers" or "only new sign-ups".
  4. Comparison: against what? "No coupon", "a 5% coupon", "business as usual".
  5. Time frame: when the outcome is measured, and when the treatment starts.

"Do coupons work?" is not a question. "Among active customers, how much more would they spend in 30 days if sent a 10% coupon than if sent nothing?" is.

A precise causal question

Treatment What we change
Outcome What we measure
Population Who it is for
Comparison Versus what

Add a time frame and the question is complete.

1 / 2

Estimand, estimator, estimate

Three words that are easy to mix up:

Term Meaning Example
Estimand The target quantity in the real world The average effect of the coupon on all customers
Estimator A recipe that uses data to approximate it Difference in group means; regression; matching
Estimate The number you get from one dataset +4.2 dollars

Choose the estimand first, then the estimator. Changing the estimator should not secretly change the question.

Common estimands

Using Y(1) for the outcome if treated and Y(0) if not (formal notation comes in potential outcomes):

  • Average treatment effect (ATE): the mean of Y(1) - Y(0) over the whole population.
  • Average effect on the treated (ATT): the same, over people who actually got the treatment.
  • Conditional effect (CATE): the effect for a subgroup, such as age under 30.
  • Intention to treat (ITT): the effect of being offered the treatment, whether or not people took it.
  • Risk difference, risk ratio, odds ratio: ways to express effects on yes/no outcomes.
Python Same data, different effect scales

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

All three describe the same effect. A risk difference of about 0.10 sounds modest; a risk ratio of about 1.5 sounds large. Report the scale that matches the decision, and often more than one.

Whom does the answer apply to?

The population can change the answer. A discount may help price-sensitive students but not heavy buyers. If your data come from students and you want to talk about all customers, you also need an argument that the effect carries over (called transportability or external validity).

An estimand in code

Here treatment effects differ by group, so ATE and ATT are different numbers even with perfect data.

Python ATE versus ATT

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The treated group contains more young people, so the ATT sits near the large effect while the ATE averages both groups. Always say which one you report. The next lessons treat these quantities formally in ATE, ATT and ATU.

Do not write "the effect of X on Y" without saying for whom and compared with what. Different readers will fill the gaps differently.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Which is the estimator in 'we compared mean spending of coupon and no-coupon customers'?

    The estimator is the recipe applied to data; the estimand is the target quantity.
  2. 2. The ATT differs from the ATE when:

    If treatment uptake is related to effect size, the treated subgroup has a different average effect.
  3. 3. A risk difference of 0.10 and a risk ratio of 1.5 from the same data are:

    From baseline 0.20 to 0.30 the difference is 0.10 and the ratio is 1.5.
  4. 4. Which parts of a causal question are missing from: 'Does the new layout improve sales?'

    Against which layout, for whom and over what period are all unspecified.

Summary

  • State treatment, outcome, population, comparison and time frame before analysis.
  • The estimand is the target, the estimator the method, the estimate the number.
  • ATE, ATT, CATE and ITT answer different questions and can differ numerically.
  • Choose the effect scale (difference, ratio, odds) that fits the decision.

Next: Simulating Causal Data