Skip to content
Tutorials Center
My learning

Topics

Lesson 31 of 37 ยท Uplift and Heterogeneous Effects

Uplift Modeling and Evaluation

Learn uplift modeling for marketing and retention: the four customer types, building an uplift model from a randomized test, and evaluating it with decile tables and cumulative gain curves.

A marketing team has a budget for 100,000 coupons and a customer list of one million. A classic response model finds the customers most likely to buy. But many of them would have bought anyway, so the coupon is wasted margin. What you want is the customers who buy because of the coupon. That is uplift modeling: predicting the incremental effect of an action for each customer, which is exactly the CATE for a binary outcome.

The four customer types

Every customer falls into one of four groups, based on what they would do with and without the offer. You never observe both, which is the fundamental problem of causal inference.

Type Buys without offer Buys with offer What to do
Persuadable No Yes Target them
Sure thing Yes Yes Skip: the offer costs money for nothing
Lost cause No No Skip
Sleeping dog Yes No Avoid: the offer backfires

Uplift for a customer is P(buy | offer) - P(buy | no offer). Persuadables have positive uplift, sleeping dogs have negative uplift, and the other two are near zero.

flowchart TD
    A["Customer"] --> B{"Buys without offer?"}
    B -->|"Yes"| C{"Buys with offer?"}
    B -->|"No"| D{"Buys with offer?"}
    C -->|"Yes"| E["Sure thing"]
    C -->|"No"| F["Sleeping dog"]
    D -->|"Yes"| G["Persuadable"]
    D -->|"No"| H["Lost cause"]

Because each customer reveals only one branch, groups are estimated, never seen directly.

Data: a randomized test

Uplift models need a randomized experiment (or a very trustworthy observational design): a random subset received the offer, the rest did not. Below we simulate one where purchase probability depends on a "loyalty" score and a "price sensitivity" score.

Python Training a T-learner uplift model

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The overall effect is small because positive and negative effects offset one another. The model finds a substantial group of "sleeping dogs" with negative predicted uplift.

Evaluating an uplift model

You cannot compare a prediction with a true individual uplift, so you evaluate on a held-out randomized sample. The standard tools:

Decile table. Rank test customers by predicted uplift and cut into 10 equal groups. In each, compute the observed treated purchase rate minus the control rate. A good model shows the largest observed uplift in the top deciles, falling and even turning negative in the bottom ones.

Cumulative gain (Qini-style) curve. Sort by predicted uplift and, for the top k customers, compute the incremental purchases n_treated_k * (rate_treated_k - rate_control_k), scaled to the treated count. Plot against k. A curve above the straight line to the overall effect (random targeting) is useful; the area between them is the AUUC or Qini coefficient.

Python Decile table and cumulative gain on held-out data

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The top deciles show clearly positive observed uplift and the bottom ones are near zero or negative. Targeting the top 30% captures most of the total incremental purchases (compare it with the "random targeting" column, which assumes gains grow in a straight line), and the curve peaks around the top half and then slips because the last customers on whom the offer backfires cancel some gains. In other words, you can send far fewer offers and lose little or nothing. Small differences between neighboring deciles are noise: each decile holds only 800 test customers.

From scores to decisions

Uplift is only useful when attached to economics. For a customer with predicted uplift u, margin per purchase m and offer cost c, treat if u * m > c. The best cutoff is therefore not "the top 30%" but "everyone whose expected incremental profit is positive". Validate the policy with a randomized hold-out: treat by the model in one random arm, treat by the old rule or nobody in another, and compare profit.

Practical cautions:

  • Uplift signals are weak and noisy; you need large experiments (tens of thousands of customers).
  • Do not train on the same data you use for the decile table.
  • Customers' behavior changes over time, so re-test the model regularly.
  • If the outcome is rare, consider a continuous proxy or a longer window.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Which customer type should NOT receive an expensive offer because they would buy anyway?

    Sure things buy with or without the offer, so the offer only costs money.
  2. 2. What is uplift for a customer?

    Uplift is the incremental purchase probability caused by the offer.
  3. 3. How do you evaluate an uplift model?

    Individual effects are never observed, so you compare observed treated-versus-control differences within groups of predicted uplift.
  4. 4. When should a customer be targeted, given margin m and cost c?

    The decision rule is expected incremental profit greater than the cost of the offer.

Summary

  • Uplift modeling predicts the incremental effect of an action; persuadables are the target, sleeping dogs should be avoided.
  • Train on randomized data with a T-learner or other meta-learner.
  • Evaluate with decile tables and cumulative gain curves on held-out randomized data.
  • Turn uplift into decisions using margin and cost, and confirm with a randomized hold-out.

Next: Doubly Robust Estimation