Skip to content
Tutorials Center
My learning

Topics

Lesson 39 of 40 ยท Projects and Practice

Project: Customer Segmentation

Segment customers with scikit-learn: build RFM features from transactions, scale and cluster with k-means, choose k, validate stability, name the segments and visualize them with PCA.

A shop with thousands of customers cannot write a personal plan for each one, but it can write five. Customer segmentation groups customers by behavior so marketing, support and pricing can treat each group differently. There are no labels to predict, so this is an unsupervised project, and the quality of the result depends on judgment as much as on numbers.

You will build RFM features (recency, frequency, monetary value) from a transaction log, cluster them with k-means, test whether the clusters are stable, and turn them into named segments that a business person could use.

Step 1: Transactions to customer features

The data is generated in code: each customer belongs to a hidden behavior type, and transactions are drawn from it. (Real data would arrive as one row per purchase.)

Python From purchases to RFM

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

RFM features are usually right-skewed: a few customers spend hugely. K-means uses distances, so before clustering we should compress the tails with a log transform and then standardize.

RFM in one slide

Recency Days since the last order.
Frequency How many orders.
Monetary How much they spent.

Simple, interpretable, and powerful for retail and subscriptions.

1 / 2

Step 2: Transform, scale and choose k

Python Choosing the number of segments

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The silhouette is highest at k = 2 (big spenders versus everyone else), because two blobs are always easier to separate than four. That is a reason to look further, not to stop: the business wants more detail than "valuable or not". Look for a peak or an elbow, then use judgment: a marketing team cannot act on twelve segments, and two segments are too coarse. Four or five is usually practical, so check that the chosen k also tells a clear story in the next step.

Step 3: Fit, profile and name the segments

Profile each cluster in the original units (not the scaled ones) so people can read it.

Python Profiling the clusters

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Read the profile like a marketer: a small segment with high spend, many orders and low recency days is your champions; a large segment with low spend, few orders and high recency days is lapsed or at risk. Give each a name and an action: thank and reward champions, nudge regulars to upgrade, win back lapsed customers with a discount.

Step 4: Validate stability and visualize

If a different seed or a resampled dataset produces very different clusters, the segmentation is not reliable. adjusted_rand_score compares two clusterings (1 = identical).

Python Stability and a PCA picture

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

An average ARI close to 1 means the segments reappear whatever sample you draw. Values around 0.5 would tell you the boundaries are arbitrary, and you should try fewer segments or different features. The PCA plot shows whether the groups are visually separate or form one continuous cloud; with real customers it is usually a cloud, and segments are a useful partition rather than natural islands.

Step 5: Use and maintain the segments

  • Assign new customers with the same fitted pipeline and km.predict (see saving models): keep the transformer and k-means together in one make_pipeline.
  • Recompute regularly: behavior drifts; refit monthly or quarterly and compare segment sizes.
  • Try alternatives from hierarchical clustering and DBSCAN or a Gaussian mixture, which gives soft membership.
  • Add features: product categories, discount usage, return rate; reduce with PCA if there are many.
  • Close the loop: A/B test an action per segment, and measure whether it paid off.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why apply log1p before scaling RFM features?

    Log compresses long tails so extreme spenders do not overwhelm every other customer.
  2. 2. Why profile clusters using the original units rather than scaled values?

    A segment description such as 'median 12 orders, 30 days since last purchase' is meaningful to the business.
  3. 3. A bootstrap stability check gives an average ARI of 0.45. What does it suggest?

    Low agreement across resamples means the partition is fragile.
  4. 4. How do you assign a new customer to a segment?

    Reuse the fitted pipeline so the new row is scaled exactly like the training data.

Summary

  • Turn raw transactions into one row per customer with RFM features.
  • Log-transform and standardize, then pick k with silhouette scores and practical judgment.
  • Profile in original units, name the segments, and attach an action to each.
  • Check stability with resampling and keep the preprocessing and k-means together in one fitted pipeline.

Next: Practice Exercises