Skip to content
Tutorials Center
My learning

Topics

Lesson 25 of 28 ยท Projects

Project: Customer Churn

Analyze why customers leave using logistic regression in Statsmodels: build features, read odds ratios and marginal effects, pick a threshold and evaluate with a confusion matrix.

A subscription company wants to know why customers cancel and which customers are likely to cancel next month, so the retention team can call the right people. This is a binary outcome (churned or not), so logistic regression is the natural tool. You will practice the full path: data, model, interpretation, decision threshold and honest evaluation.

The data are simulated from a known recipe: new customers churn more, support tickets raise churn, annual contracts and higher engagement lower it.

Step 1: data and base rate

Python Create the data and look at the churn rate

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The overall churn rate (the base rate) matters: a model that always predicts "stays" would be right most of the time if churn is rare. Accuracy alone will therefore mislead.

Step 2: fit and interpret

We split off a test set first, so every number reported later is honest.

Python Fit the logistic model and read odds ratios

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Interpretation for the business (always say "holding other variables fixed"):

  • Each extra support ticket multiplies the odds of churning by about 1.4.
  • Each additional month of tenure lowers the odds by roughly 4%.
  • Annual contracts cut the odds to about a third of monthly contracts.
  • More logins means lower churn.

See Logistic and Probit Regression for how to read these numbers.

Step 3: marginal effects

Executives prefer probabilities to odds. Average marginal effects translate each coefficient into "percentage points of churn probability".

Python Average marginal effects

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

For example: "one more ticket in the last quarter raises the churn probability by about 6 percentage points on average".

Step 4: choose a threshold and evaluate

A probability becomes a decision only through a threshold. The retention team can call only the highest-risk customers, so the threshold is a business choice. Evaluate on the test set with a confusion matrix, precision (of those flagged, how many really leave) and recall (of those who leave, how many we flagged).

Python Threshold trade-offs on the test set

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Lower thresholds catch more leavers (higher recall) but waste calls on loyal customers (lower precision). If a retention offer is cheap, favor recall; if each call is expensive, favor precision.

Step 5: calibration check

A good probability model is calibrated: among customers given a 30% risk, about 30% should actually leave. Group the test predictions into bands and compare the average prediction with the actual rate.

Python Calibration table

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The two columns should track each other. If they do not, consider interactions, transformations such as np.log(tenure), or a different model.

Report and next steps

  • Drivers: tickets raise churn, tenure, engagement and annual contracts reduce it (with odds ratios and intervals).
  • Targeting: the top risk bands and a threshold chosen for the call budget.
  • Caveats: association, not causation; an annual contract may be chosen by customers who were already loyal. A randomized retention offer would measure the true effect.
  • Extensions: add interaction contract * tickets; handle imbalance with weights; try regularization (Regularized Regression); compare with sm.Probit.

Check your understanding

Quick check ยท 3 questions

of 3 answered You scored / 3 ยท perfect!

  1. 1. Why is accuracy a poor sole metric for a churn model?

    With an imbalanced outcome, accuracy rewards ignoring the minority class; use precision, recall and calibration.
  2. 2. What happens when you lower the classification threshold from 0.5 to 0.3?

    A lower bar flags more people, catching more true leavers but also more false alarms.
  3. 3. What does it mean that a model is calibrated?

    Among cases predicted at 30% risk, about 30% should actually have the event.

Summary

  • Churn is a binary outcome: fit a logit, report odds ratios and marginal effects.
  • Choose the threshold from business costs; judge with precision, recall and calibration on held-out data.
  • Do not claim causation from observational drivers.

Next: Project: Sales Forecast