Skip to content
Tutorials Center
My learning

Topics

Lesson 20 of 37 ยท Matching and Weighting

Nearest-Neighbor Matching

Build matched samples with nearest-neighbor matching on covariates or propensity scores, add calipers, and estimate the effect on the treated (ATT) with runnable Python.

Matching is the most intuitive causal method: for every treated unit, find a control unit that looks the same, and compare their outcomes. If the two really are alike on everything that matters, their difference in outcomes is a fair estimate of what the treatment did to the treated unit. Repeat for everyone and average.

This lesson builds matching from scratch with NumPy and scikit-learn, so you see exactly what the libraries do. It assumes you know the propensity score.

What are we estimating?

Matching naturally estimates the average treatment effect on the treated (ATT): for each treated unit you search for a counterfactual among the controls. The effect for the whole population (ATE) would also need matches for each control unit, which is possible but less common. ATT answers "what did the program do for the people it actually reached?", which is often exactly the business question.

Matching on the propensity score

Distance on one number is easy: sort or use a nearest-neighbor search on the score. Two design choices matter.

  • With or without replacement. With replacement, one control can serve many treated units. That gives better matches and no order dependence, but fewer distinct controls, so the variance is higher.
  • Number of neighbors. One-to-one matching minimizes bias; matching each treated unit to k controls and averaging lowers variance.
Python 1-to-1 propensity matching (ATT)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The matched estimate is much closer to 2.0. The last line reports how far apart the matched scores are on average; it should be tiny.

Calipers: refusing bad matches

Nearest does not mean close. If a treated unit has a score of 0.95 and the best control has 0.80, the pair is a poor comparison. A caliper sets a maximum allowed distance, commonly 0.2 standard deviations of the logit of the score. Treated units without a match inside the caliper are dropped, and you report that the estimate applies to the units that could be matched.

Python Adding a caliper and matching on the covariates themselves

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Matching directly on covariates uses Euclidean distance, so variables with large scales dominate. Standardize them first, or use Mahalanobis distance, which accounts for scale and correlation. With many covariates, exact or distance matching suffers from the curse of dimensionality, which is the main reason people match on the propensity score instead.

Practical advice

flowchart TD
    A["Estimate the score"] --> B["Match with a caliper"]
    B --> C["Check balance"]
    C -->|"Not balanced"| A
    C -->|"Balanced"| D["Compare outcomes"]

Balance is checked before looking at outcomes. Never tune the matching to get the effect you like.

  1. Decide the design without the outcome. Only covariates and treatment guide your choices.
  2. Check balance after matching. If it is poor, change the score model or the caliper.
  3. Mind your standard errors. Naive standard errors ignore the matching step. Use the bootstrap or a library that implements the Abadie-Imbens variance.
  4. Report what was dropped. Units without matches change the population your estimate describes.

King and Nielsen argue that matching on the propensity score alone can increase imbalance and model dependence compared to other distances when matching is pruned aggressively. Treat the score as one tool and always inspect covariate balance.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Matching each treated unit to controls naturally estimates which quantity?

    Each treated unit gets a counterfactual from the control pool, so the result describes the treated units.
  2. 2. What does a caliper do?

    Pairs farther than the caliper are poor comparisons, so those treated units are left out.
  3. 3. What is a benefit of matching with replacement?

    Reusing controls gives better matches but fewer distinct controls, which raises variance.
  4. 4. When should you choose caliper and score-model settings?

    Tuning on the outcome leaks your hopes into the estimate. Use balance as the guide.

Summary

  • Matching pairs each treated unit with similar controls and averages the outcome differences; it estimates the ATT.
  • Match on the propensity score, or on standardized/Mahalanobis-scaled covariates.
  • Use calipers to discard bad matches and report how many units were dropped.
  • Choose settings by covariate balance, never by the outcome.
  • Use the bootstrap or a dedicated variance estimator for standard errors.

Next: Inverse Probability Weighting