Skip to content
Tutorials Center
My learning

Topics

Lesson 35 of 37 ยท Projects and Practice

Project: Coupon Targeting with Uplift

End-to-end project: analyze a randomized coupon test, check balance, estimate the average effect, train an uplift model, evaluate the targeting policy's profit and write a recommendation.

In this project you play the data scientist at an online shop. Marketing ran a randomized test: half of 30,000 customers received a 10% coupon, half did not. The coupon costs margin, so the question is not "does the coupon increase purchases?" but "who should get it so that profit goes up?"

You will walk through a complete causal workflow: check the experiment, estimate the average effect, discover heterogeneity, build a targeting policy and evaluate it honestly. The methods come from the lessons on heterogeneous effects, meta-learners and uplift evaluation.

The business setup

  • Margin per purchase if the customer buys: 20 (before the coupon).
  • A coupon redemption costs 6 (the discount), paid only when the customer buys with it. To keep things simple we charge the coupon cost to every treated customer who buys.
  • Profit per customer = 20 * bought - 6 * (bought and treated).

So a coupon only pays off for a customer if the extra purchases it causes outweigh the discount given to purchases that would have happened anyway. The exact rule is in step 2; we let the data tell us who qualifies.

Step 1: sanity-check the experiment

Before trusting any effect, verify that randomization worked: pre-treatment covariates should look the same in both arms.

Python Step 1: balance and the average effect

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The covariates are balanced, which is what randomization promises. The coupon raises purchases but the profit per customer changes by much less (it might even be negative) because every purchase made with a coupon now earns 6 less. Treating everybody is not obviously a good idea.

Step 2: find who responds

Now split the data in two. We train the model on 60% and evaluate everything on the other 40%, so that evaluation is not flattered by overfitting. We use a T-learner with gradient boosting classifiers and then measure how much profit a targeting rule would produce.

The rule: treat customer x if the predicted extra purchase probability times the margin is larger than the discount paid on purchases the customer would have made anyway. With p1 = P(buy | coupon) and p0 = P(buy | no coupon):

expected profit with coupon = 14 * p1, expected profit without = 20 * p0

so treat if 14 * p1 > 20 * p0.

To evaluate a policy on randomized data, use inverse probability weighting: only customers whose actual arm matches what the policy would have done contribute, each weighted by 1 / 0.5.

value(policy) = mean( 1[t == policy(x)] * profit / 0.5 )

Python Step 2: uplift model, policy and its value

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Treating nobody beats treating everyone, and the model policy beats both, though by a modest margin (a few tenths per customer), which is why step 3 matters. The final table shows the model discovered the responsive segment on its own: it targets most discount lovers and almost none of the other customers.

Step 3: check it is not luck

The evaluation set is a finite sample, so the policy value has a standard error. A responsible report gives an interval. You can bootstrap the test customers: resample them, recompute the value difference between the model policy and "treat nobody", and look at the spread. Always write the comparison as a difference: both policies are evaluated on the same customers, which cancels most noise.

Python
rng = np.random.default_rng(0)
diffs = []
for _ in range(200):
    idx = rng.integers(0, len(test), len(test))
    s = test.iloc[idx]
    pol = s.treat_by_model.to_numpy()
    v_model = np.mean((s.t.to_numpy() == pol) * s.profit.to_numpy() / 0.5)
    v_none = np.mean((s.t.to_numpy() == 0) * s.profit.to_numpy() / 0.5)
    diffs.append(v_model - v_none)
print(np.percentile(diffs, [2.5, 50, 97.5]))

If the interval stays above zero, you can recommend the policy with confidence. If it covers zero, say so and propose a bigger or longer test.

Step 4: write the recommendation

A good causal report is short and states assumptions. A template:

  1. Question and decision. Which customers should receive the 10% coupon?
  2. Design. Randomized 50/50 test, 30,000 customers; balance verified.
  3. Findings. The coupon raises purchase rate overall, but profit per customer falls when everyone is treated. Effects are concentrated in discount-loving customers; for the rest the discount costs more than it earns.
  4. Recommendation. Send coupons to the customers the model selects (about a quarter of the base); keep a 10% random hold-out to monitor.
  5. Risks. The model was trained on one season; behavior shifts. Re-test quarterly. Short-term purchase effects may differ from long-term loyalty effects.
flowchart LR
    A["Randomized test"] --> B["Balance check"]
    B --> C["ATE and profit effect"]
    C --> D["Uplift model on train split"]
    D --> E["Policy value on test split"]
    E --> F["Interval by bootstrap"]
    F --> G["Recommendation and hold-out"]

The whole project in one line.

Extensions to try

  • Replace the T-learner with an X-learner or add more depth and compare policy values.
  • Add a cost for sending the coupon (email) and find the best targeting share.
  • Estimate the profit curve: treat the top k% by predicted uplift for k from 0 to 100 and plot the value of each.
  • Simulate a non-randomized version where discount lovers were more likely to receive the coupon, and fix it with IPW or AIPW.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why evaluate the targeting policy on a split that the model did not train on?

    A flexible model can fit noise, so honest policy values need fresh data.
  2. 2. How do you estimate a policy's value from a randomized test?

    Those customers were randomly assigned to the policy's action, so reweighting gives an unbiased estimate of what the policy would earn.
  3. 3. The coupon increases purchases but treating everyone lowers profit. Why?

    Sure things and sleeping dogs make the coupon unprofitable overall.
  4. 4. What should always accompany the policy value in a report?

    Decision-makers need to know how much the estimate could move and how it will be monitored.

Summary

  • Start every analysis by checking the randomization and the average effect on the business metric, not just on purchases.
  • Train an uplift model on one split and evaluate a targeting policy on another with inverse probability weighting.
  • Report differences between policies with a bootstrap interval.
  • Close with a recommendation, risks and a hold-out to monitor.

Next: Project: A Policy Change with Difference-in-Differences