Skip to content
Tutorials Center
My learning

Topics

Lesson 44 of 76 ยท Statistics for Data Science

Causal Thinking and Confounding

Learn why correlation is not causation: confounders, colliders, DAG diagrams, randomization versus observational data, and regression adjustment with live Python.

"Customers who use the new feature spend more." That sentence is easy to compute and easy to misread. Does the feature make people spend more, or do big spenders simply love new features? The answer decides whether you invest in promoting the feature or do nothing. Data alone, without a way of thinking about how it was generated, cannot tell the two stories apart.

Causal thinking is the habit of asking "what would have happened to the same people if we had acted differently?" This lesson introduces the vocabulary (confounders, colliders, causal diagrams), shows with simulations how a naive comparison can be off by a factor of five, and sketches the main ways to get closer to a causal answer when you cannot run an experiment. Where you can randomize, A/B testing is the gold standard; this lesson is about everything else.

Correlation, association and causation

Two variables are associated if knowing one helps you predict the other. That is all a correlation says. A causal effect is the change in an outcome that would result from intervening on a cause, holding everything else fixed. The two differ for three main reasons:

  • Confounding: a third variable influences both the supposed cause and the outcome.
  • Reverse causation: the outcome influences the supposed cause (people who feel unwell take more medicine).
  • Selection: the data you see were filtered in a way related to both variables.

Predicting and explaining are different jobs. A model that uses ice-cream sales to predict drownings may predict well on a warm day, but banning ice cream will save no one. Whenever your question starts with "what happens if we change...", you need a causal argument.

Drawing the story: causal diagrams

A causal diagram (a directed acyclic graph, or DAG) draws each variable as a node and each assumed direct cause as an arrow. It forces you to state your assumptions, which is half the battle. Three small patterns explain most of causal analysis.

flowchart TD
    subgraph Confounder
    Z1["Engagement"] --> X1["Uses feature"]
    Z1 --> Y1["Spend"]
    X1 --> Y1
    end
    subgraph Collider
    X2["Talent"] --> C2["Fame"]
    Y2["Charm"] --> C2
    end

Left: engagement is a common cause (confounder) of using the feature and of spending. Right: fame is a common effect (collider) of talent and charm.

  • A confounder is a common cause. It creates a misleading association between two variables that do not affect each other (or exaggerates a real one). To estimate the effect of the feature, you must account for engagement.
  • A collider is a common effect. Its two causes are independent, but if you select or control for the collider, you create a false association between them.
  • A mediator sits on the path from cause to outcome (feature, then more sessions, then more spending). Controlling for a mediator blocks part of the very effect you want to measure.

The rule of thumb: adjust for confounders, do not adjust for colliders or mediators (unless you are asking about the direct effect only). The diagram is how you tell them apart, and it comes from domain knowledge, not from the data.

Confounding in action

Here users decide for themselves whether to opt in to a new feature. Engaged users opt in more often, and engaged users also spend more. The true causal effect of the feature is +2 per week, because we built the simulation that way. A naive comparison of the two groups will not find it.

Python A naive comparison versus adjustment

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The naive difference is about 10.4, five times the real effect, because the opt-in group is simply a more engaged group. Two fixes recover the truth:

  • Regression adjustment: include the confounder as another predictor. The coefficient on opt_in is then the difference at the same level of engagement. It gives 1.83 with an interval that contains the true 2.0.
  • Stratification: split users into bands of similar engagement, compare treated and untreated within each band, and average. The within-band differences are all small and positive compared with the naive 10. With only five bands the answer (2.72) is still a bit off, because users inside a band still differ in engagement. Finer bands, or a model, reduce this leftover bias.

A related idea is matching: for each treated user find an untreated user with similar characteristics and compare the pairs. Matching and stratification are the same spirit as adjustment, just without a model formula.

All of these share one limit. They only adjust for confounders you measured. If something unmeasured (say, a user's income) also drives both opt-in and spending, the estimate stays biased, and no amount of clever statistics will show it. That is why causal claims from observational data are always conditional on assumptions, and why you should write the assumptions down.

Why randomization is different

In an experiment, a coin flip, not the user's own choice, decides who gets the feature. The coin knows nothing about engagement, income or any unmeasured factor, so on average all of them are balanced between the groups. The simulation below runs the same world twice, once with self-selection and once with random assignment.

Python Self-selected versus randomized assignment

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The last block explains the whole story: with self-selection, treated users are about one full standard deviation more engaged than untreated users; with random assignment the groups are the same. Randomization buys comparability for free, including for the variables you never thought of. When you can randomize, do.

When you cannot (ethical limits, too expensive, the past is already done), look for natural experiments where something random-like decided exposure: a policy that applied to one region only, a lottery, a cutoff score. The methods built on them (difference-in-differences, regression discontinuity, instrumental variables) are advanced topics, but they all aim to recreate the coin flip.

Colliders: when adjusting makes things worse

It is tempting to think "more controls is always better". Colliders show why it is not. Suppose talent and charm are unrelated in the population, but both help people become famous. If you look only at famous people, or control for a fame score, talent and charm suddenly appear negatively related: among the famous, someone with little talent must have had plenty of charm to get there.

Python Selecting on a collider invents a relationship

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The correlation is near zero for everyone and strongly negative among the famous, and the regression coefficient jumps to about -0.8 once the collider is controlled. Real examples are everywhere: studying only hospitalized patients, only startups that survived, only customers who complained. This is the same family of problems as the selection and survivorship bias in sampling and collection bias.

A practical workflow

  1. State the causal question with a clear treatment, outcome and population: "What is the effect of opting in on weekly spend for active users?"
  2. Draw the diagram with domain experts. Mark confounders, mediators and colliders.
  3. Prefer an experiment. If impossible, pick an observational design (adjustment, matching, natural experiment).
  4. Adjust for confounders, not for colliders or mediators. Measure the confounders well.
  5. Check sensitivity: how strong would an unmeasured confounder have to be to erase the result? Report that honestly.
  6. Word the conclusion carefully. "Associated with" for correlations, "estimated effect, assuming no unmeasured confounding" for adjusted estimates.

Note: Machine-learning accuracy does not buy causal validity. A very accurate model that predicts spending from "opted in" is still just reporting an association. Causal claims need a design, not a better algorithm.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Which variable is a confounder in the question 'does coffee cause heart disease?'

    A confounder is a common cause of the exposure and the outcome. If smokers drink more coffee and also have more heart disease, the raw association is partly smoking.
  2. 2. Why does randomizing the treatment solve the confounding problem?

    Random assignment breaks any link between pre-existing characteristics and the treatment, so only the treatment differs between groups, on average.
  3. 3. You control for a variable that is caused by both your treatment and your outcome. What is likely to happen?

    Conditioning on a collider links its two causes, creating an association that was not there before.
  4. 4. Regression adjustment for measured confounders is enough when...

    Adjustment only removes bias from confounders that are in the model. Unmeasured confounders remain a threat, so the assumption must be argued, not assumed.
  5. 5. A mediator lies on the path from treatment to outcome. Should you normally adjust for it when estimating the total effect?

    Part of the treatment's effect works through the mediator. Controlling for it measures only the remaining direct effect.

Summary

  • Association is not causation: confounding, reverse causation and selection can all produce correlations without effects.
  • A causal diagram states your assumptions; confounders are common causes, colliders are common effects, mediators sit on the path.
  • Adjust for confounders (regression, stratification, matching); do not adjust for colliders, and think before adjusting for mediators.
  • Observational adjustments only handle measured confounders, so the conclusion rests on stated assumptions.
  • Randomization balances everything, measured or not; use it whenever you can, and look for natural experiments otherwise.

Next: Bayesian Thinking for Analysts, a different way to express uncertainty about an effect.