Skip to content
Tutorials Center
My learning

Topics

Lesson 53 of 76 ยท Modelling

Clustering and Segmentation

Group customers without labels: scale features, run k-means and hierarchical clustering, choose k with inertia and silhouette, and profile each segment.

Most models in this chapter learn from a column you want to predict. Clustering is different: nobody tells the algorithm what the groups are. You hand it a table, it looks for rows that resemble each other, and it returns a group number for each row. This is called unsupervised learning, and its most common business use is segmentation: splitting customers, products or sessions into a handful of groups that can be treated differently.

Clustering is easy to run and easy to misuse. The algorithm will always return groups, even when the data has no real groups at all. So the skill is not calling fit; it is preparing the features, choosing how many groups to ask for, checking that the result is stable, and turning numeric labels into a description a colleague can act on. This lesson covers all four steps with k-means and hierarchical clustering. If you have not yet seen scaling, read Encoding and Scaling Features first.

The data: customers with three hidden groups

To judge a clustering method honestly you need to know the right answer. So we build a synthetic customer table with three hidden groups (bargain hunters, regular customers and big spenders) and then pretend we do not know the group of each row. The three features are days since the last order, number of orders, and total spend. The generator uses a fixed seed, so every run gives the same table.

flowchart LR
    A["Customer table"] --> B["Choose and scale<br/>features"]
    B --> C["Run algorithm<br/>for several k"]
    C --> D["Compare k<br/>(inertia, silhouette)"]
    D --> E["Profile the<br/>segments"]
    E --> F["Name them and<br/>test usefulness"]

A clustering project is mostly work before and after the algorithm call.

Why scaling comes first

Both k-means and hierarchical clustering are built on distance. If one feature is measured in dollars (hundreds) and another in orders (a dozen), the dollars decide almost everything and the other column barely counts. Standardizing each feature to mean 0 and standard deviation 1 gives every column a fair vote. The next example clusters the same table twice, once raw and once scaled, and compares each result with the hidden truth using the adjusted Rand index (ARI): 1.0 means the same grouping, values near 0 mean no better than chance.

Python Scaling changes the clusters

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

With raw features the spend column behaves like the only feature: the algorithm wastes one of its three clusters on splitting the big spenders into a "high" and a "very high" group (centres 0 and 2 above), and the agreement with the truth drops from 0.59 to 0.42. Scaling is not optional here; make it a fixed step in your pipeline. For skewed columns like spend, a log transform before scaling often helps too (see Feature Engineering).

k-means: how it works

k-means repeats two steps. First, every row joins the nearest of k centroids (average points). Second, each centroid moves to the mean of the rows that joined it. It stops when nothing changes. The result depends on the starting centroids, so scikit-learn runs several starts (n_init) and keeps the best one. Always set random_state so your result is reproducible.

flowchart TD
    S["Pick k starting centroids"] --> A["Assign each row to<br/>its nearest centroid"]
    A --> M["Move each centroid to<br/>the mean of its rows"]
    M --> Q{"Did any<br/>assignment change?"}
    Q -- "yes" --> A
    Q -- "no" --> D["Done: labels and centroids"]

k-means alternates between assigning rows and moving centroids until the assignments settle.

k-means makes assumptions worth remembering. It prefers round, similarly sized blobs, because it uses plain distance to a centre. It works poorly on long curved shapes, groups of very different sizes or density, and data with many outliers, which drag centroids away from the bulk. Every row is forced into some group, including rows that belong nowhere.

Choosing k

You must pick k before you start. Two common helpers:

  • Inertia is the sum of squared distances from each row to its centroid. It always falls as k grows (more centroids means shorter distances), so you look for an elbow, the point after which extra groups give little gain.
  • Silhouette score compares, for each row, how close it is to its own cluster against the nearest other cluster. It ranges from -1 to 1; higher means tighter, better separated groups. Unlike inertia, it can fall when k is too large, so its maximum is a candidate for k.
Python Elbow and silhouette for k = 2 to 7

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Read the table with care. The silhouette is highest at k = 2, because two groups (the big spenders and everybody else) are the cleanest split. Inertia drops steeply until k = 3 and then flattens, and we happen to know that three groups is the truth. When metrics disagree, do not hunt for a magic formula. Treat them as evidence and add the questions that matter for the business: can the marketing team run three different campaigns? Does each group have enough customers? Do the groups stay the same on a second sample?

Silhouette and inertia measure geometry, not usefulness. A clustering with a lower score but segments you can name and act on can be the better choice.

Profiling the segments

A label such as "cluster 1" means nothing to a manager. Profiling turns labels into descriptions: group the original, unscaled table by cluster and compare size and averages. Always profile in the real units, because mean spend of 126 dollars is readable and a z-score of 0.3 is not. Also compare the clusters with the hidden truth, which we can do only because this data is synthetic; with real data you have no such check, which is why stability and usefulness matter.

Python Cluster profiles in business units

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The profile table suggests names: a small group that orders often and spends most (about a quarter of the customers but well over half the revenue), a middle group, and a lapsed, low-spend group. The cross-tab shows the algorithm recovers the big spenders perfectly but blurs bargain and regular customers, because in the generator their behaviour overlaps. That is typical of real data: boundaries between segments are soft, and a customer near the border could easily have landed on the other side. Treat segments as a convenient simplification, not as natural kinds.

Cluster numbers are arbitrary. Refit with another seed and "cluster 0" may become "cluster 2". Name the segments by their profile and never by their number.

Hierarchical clustering

Hierarchical (agglomerative) clustering starts with every row as its own cluster and repeatedly merges the two closest clusters until one remains. The merge history is a tree, drawn as a dendrogram; cutting the tree at a height gives any number of clusters, so you can look at the structure before committing to k. The linkage rule defines "closest": ward merges the pair that increases within-cluster variance least and tends to give compact, balanced clusters, so it is a good default; average and complete use distances between members; single follows chains and can string unrelated points together.

The price is cost: the algorithm needs distances between all pairs of rows, which grows with the square of the row count. It suits thousands of rows, not millions. A practical trick is to cluster a sample, or to run k-means with a large k first and cluster the centroids.

Python A dendrogram and a comparison with k-means

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The long vertical lines at the top of the dendrogram are the big merges: cutting just below them gives the natural number of groups. The two methods agree on the clear structure (both isolate the same 120 big spenders) and disagree on the border between the overlapping groups; their mutual ARI is 0.72, and ward matches the hidden truth slightly worse (0.47) than k-means (0.59) here. Agreement between different algorithms is a useful sanity check; large disagreement warns you that the segments are fragile.

Divisive clustering and the limits of hierarchies

Agglomerative clustering works bottom-up. The mirror image is divisive clustering, which works top-down: start with all rows in one cluster, split it into two, then keep splitting the cluster that is worst (for instance the one with the largest within-cluster sum of squares) until you reach the number of clusters you want or every row is alone. A classic implementation of one split is to run 2-means inside the cluster, which is exactly what scikit-learn's BisectingKMeans does. Divisive methods can be cheaper than agglomerative ones when you only want a few clusters, because they never build all pairwise merges, and they see the global structure first. They are less common because the number of possible splits is huge, so every implementation relies on a heuristic.

Python Divisive clustering with bisecting k-means

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Each split lowers the total within-cluster error, and the table of sizes shows the tree growing from the top. Bisecting k-means finds nearly the same groups as plain k-means here, a little less exactly because an early split is never revisited.

Drawbacks of hierarchical clustering (both directions):

  • Greedy and irreversible. A merge (or split) is never undone, so an early mistake stays in the tree.
  • Cost. Memory and time grow roughly with the square of the number of rows, which rules out very large data.
  • Sensitivity. Noise and outliers change the tree; single linkage chains clusters together, and ward assumes roughly round clusters and Euclidean distance.
  • Choosing the cut. The dendrogram does not tell you the right height; you still choose it with a silhouette score or domain sense.
  • Scale and distance. Features must be scaled first, exactly as for k-means.

On the other hand you get a whole family of solutions at once, no random starts, and a picture that explains how groups relate: that is why hierarchies remain popular for small and medium data and for biology-style trees.

Checking that clusters are real

Before presenting segments, ask three questions:

  1. Stability. Refit on a bootstrap sample or with another seed. Do most customers keep their segment?
  2. Separation. Do the segment profiles differ on features you did not cluster on, such as churn rate or channel?
  3. Usefulness. Would a different action be taken for each segment? If two segments get the same treatment, merge them.

Beyond k-means and ward, you will meet DBSCAN, which finds dense regions and marks sparse rows as noise (useful for oddly shaped groups and outliers), and Gaussian mixture models, which give each row a probability of belonging to each group. They are worth learning once the basics are comfortable. To see high-dimensional clusters on a plot, the next lesson on dimensionality reduction shows how to project them onto two axes.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Why should features usually be standardized before k-means?

    k-means and hierarchical clustering work with distances. Without scaling, the column with the largest numbers (such as dollars) decides nearly everything.
  2. 2. What does a higher silhouette score indicate?

    The silhouette compares each row's distance to its own cluster with the distance to the nearest other cluster. It does not use true labels and does not reward a larger k.
  3. 3. Inertia always decreases as k increases. How do you use it to choose k?

    More centroids always reduce distances, so the minimum is useless. The elbow marks where adding groups stops paying off.
  4. 4. Which statement about cluster numbers is correct?

    The labels only identify groups within one fit. Another seed or sample can reorder them, so describe segments by what they contain.
  5. 5. When is hierarchical clustering a poor choice?

    It needs pairwise distances, whose number grows with the square of the row count, so very large tables call for k-means or a sample.

Summary

  • Clustering is unsupervised: it groups similar rows without a target, and always returns groups, even when none exist.
  • Scale features first, because clustering is based on distance.
  • k-means alternates between assigning rows to centroids and moving centroids; use n_init and a fixed random_state.
  • Choose k with inertia (elbow), silhouette and business usefulness together; the metrics can disagree.
  • Profile clusters in original units, name them by what they contain, and never rely on cluster numbers.
  • Hierarchical clustering gives a dendrogram and suits thousands of rows; compare methods and check stability before trusting segments.

Next: Dimensionality Reduction and Visualization of High-D Data, for seeing many features at once.