Skip to content
Tutorials Center
My learning

Topics

Lesson 19 of 37 ยท Matching and Weighting

Propensity Scores

Learn what a propensity score is, how to estimate it with logistic regression, and how stratifying on it removes confounding in observational data, with runnable Python.

When treatment was not randomized, the people who got treated usually differ from the people who did not. Patients who take a new drug may be sicker, customers who see a promotion may already be loyal, students who attend tutoring may be more motivated. If those differences also affect the outcome, a plain comparison of group averages is biased.

The propensity score is a clever way to handle many such differences at once. Instead of balancing ten or fifty covariates one by one, you compress them into a single number: the probability that a unit receives treatment, given its covariates. Comparing treated and untreated units that have the same propensity score mimics a small randomized experiment. This lesson shows how to build the score; the next lessons use it for matching and weighting.

The definition

For a unit with covariates X, the propensity score is

e(X) = P(T = 1 | X)

where T is 1 for treated and 0 for control. Rosenbaum and Rubin showed a key fact: if the treatment is independent of the potential outcomes once you condition on X (no unmeasured confounding), then it is also independent of them once you condition on e(X) alone. In words: the score is a balancing score. Units with the same score have, on average, the same distribution of covariates, whether they were treated or not.

You almost never know e(X). In an experiment it is a number you chose (for example 0.5). In observational data you estimate it, most often with logistic regression.

The propensity score fixes measured confounding only. If an important confounder is not in your data, no amount of score modeling will remove its bias. See sensitivity analysis.

A confounded dataset

Below we simulate data where we know the truth. Two covariates x1 and x2 raise both the chance of treatment and the outcome. The real treatment effect is exactly 2.0, so we can see whether a method recovers it.

Python Naive comparison versus the truth

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The naive difference is far above 2.0 because the treated group has higher x1 and x2, and those raise the outcome by themselves. The covariate table at the bottom shows the imbalance directly.

Estimating the score

Fit a classifier that predicts t from the covariates and keep the predicted probabilities. Logistic regression is the standard choice because it is stable and easy to explain. Gradient boosting or random forests can capture interactions, but they tend to produce extreme probabilities that cause trouble later.

Python
from sklearn.linear_model import LogisticRegression

model = LogisticRegression(C=1e6, max_iter=1000)   # almost no regularization
model.fit(df[["x1", "x2"]], df["t"])
df["ps"] = model.predict_proba(df[["x1", "x2"]])[:, 1]

Notice the large C: scikit-learn regularizes by default, which shrinks the score toward 0.5 and weakens balancing. For a propensity model you care about unbiased probabilities, not about out-of-sample accuracy, so you do not need to tune for AUC.

What goes into the model

Include variables that affect the outcome, because they are the real confounders, and variables that affect both treatment and outcome. Do not include anything measured after treatment (a mediator or a consequence of the treatment). That would remove part of the effect you want to measure. See common pitfalls.

Stratifying on the score

The simplest way to use the score is to cut it into groups (quintiles are traditional), compare treated and control inside each group, and average the differences. Within a narrow score band the covariates are nearly balanced, so the within-band difference is close to causal.

Python Propensity score stratification

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The stratified estimate lands much closer to 2.0 than the naive one. It is not perfect: five coarse bands leave some residual confounding inside each band. More strata reduce that, at the price of fewer units per stratum.

How the pieces fit

flowchart LR
    A["Covariates X"] --> B["Logistic model"]
    B --> C["Score e(X)"]
    C --> D["Match"]
    C --> E["Weight"]
    C --> F["Stratify"]
    D --> G["Effect estimate"]
    E --> G
    F --> G

One score, three common ways of using it.

Overlap: the other half of the story

A score is only useful if both groups exist at every score level. If some units have a score of 0.99 and no control unit has a score above 0.9, there is nobody to compare them with. This requirement is called overlap or positivity. You check it by comparing score histograms by group, and you handle violations by trimming or restricting the population. The lesson on balance and overlap shows how.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does the propensity score e(X) measure?

    e(X) = P(T = 1 | X). It models how treatment was assigned, not what happens to the outcome.
  2. 2. Why is a very large C used in the logistic regression above?

    A propensity model is judged by how well it balances covariates, so heavy regularization that biases the probabilities is unhelpful.
  3. 3. Which variable should NOT go into the propensity model?

    Post-treatment variables are consequences of the treatment. Adjusting for them blocks part of the effect you want to estimate.
  4. 4. Can a propensity score remove bias from an unmeasured confounder?

    The score depends only on observed covariates, so unobserved confounding stays.

Summary

  • The propensity score is the probability of treatment given covariates, e(X) = P(T=1|X).
  • Under no unmeasured confounding, conditioning on the score is enough to remove confounding by X.
  • Estimate it with logistic regression, with little regularization and only pre-treatment variables.
  • Stratifying on score quantiles gives a simple, transparent estimate.
  • Always check overlap: you need both treated and control units at every score level.

Next: Nearest-Neighbor Matching