Skip to content
Tutorials Center
My learning

Topics

Lesson 26 of 39 ยท Clustering

Clustering Basics

Learn what clustering is, how it differs from classification, the main method families (partitioning, hierarchical, density, grid, model-based) and why scaling and distance choices shape clusters.

Clustering groups records so that records in the same group (a cluster) are more similar to each other than to records in other groups. Nobody tells the algorithm the groups in advance: it is unsupervised learning. Typical uses are customer segmentation, grouping documents by topic, finding regions with similar crime patterns, compressing images by colour and discovering species in biological data.

Because there are no labels, there is also no single correct answer. Whether three clusters or five is "right" depends on the goal. That is the main difference from classification, where the true labels let you score the model directly.

Ingredients of a clustering

Every clustering method needs three decisions:

  1. A similarity or distance measure between records (see similarity and distance).
  2. A clustering criterion: what makes a good grouping (compact groups, connected dense regions, probability model...).
  3. A search strategy to find a good grouping, because checking all of them is impossible.

Five families of clustering

Partitioning k-means, k-medoids
Hierarchical Merge or split, dendrogram
Density-based DBSCAN, OPTICS
Grid-based STING, CLIQUE

A fifth family, model-based clustering (Gaussian mixtures, EM), fits probability distributions.

1 / 2
  • Partitioning methods split the data into k groups; you choose k. Fast and good for round, similar-sized clusters. See k-means and variants.
  • Hierarchical methods build a tree of nested clusters, either bottom-up (agglomerative) or top-down (divisive). See hierarchical clustering.
  • Density-based methods grow clusters wherever records are densely packed, so they find odd shapes and mark sparse points as noise. See density and grid methods.
  • Grid-based methods divide the space into cells and cluster the cells, which is very fast on large data.
  • Model-based methods assume the data came from a mix of distributions and estimate them, giving soft (probabilistic) memberships.

What a good clustering algorithm needs

Textbooks list requirements that guide the choice:

  • Scalability: works on millions of records.
  • Different data types: numbers, categories, text.
  • Arbitrary cluster shapes, not just spheres.
  • Little need for domain knowledge to set parameters (many methods need k or a radius).
  • Robustness to noise and outliers.
  • Order independence: results should not depend on row order.
  • High dimensions: distances become less meaningful as features multiply.
  • Interpretability: clusters should be describable to a human.

Same data, different methods

The next example generates data with three round blobs and, separately, two curved moons, then runs three methods. Round data suits k-means; curved data does not.

Python Which method fits which shape?

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

ARI (adjusted Rand index) is 1 for a perfect match with known groups, near 0 for random labels. We can use it here only because the generator gave us the truth; with real data you must use internal evaluation.

Scaling decides the result

Distance-based clustering inherits the scale of your features. If one column is salary in dollars and another is age in years, salary controls every distance.

Python Clusters before and after scaling

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Without scaling, salary noise dominates and the age groups are mixed. After standardizing, the age structure appears. Preprocessing ideas live in data transformation.

Hard versus soft clustering

In hard clustering each record belongs to exactly one cluster. In soft (fuzzy or probabilistic) clustering each record has a membership degree for every cluster, e.g. 70% cluster A, 30% cluster B. Soft memberships are valuable when groups overlap, such as customers who behave partly like two segments.

Python Soft memberships from a Gaussian mixture

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why is clustering called unsupervised?

    The algorithm discovers groups from the data alone, with no teacher supplying labels.
  2. 2. Which family can find curved, arbitrarily shaped clusters and mark noise?

    Density-based methods grow clusters through dense regions and label sparse points as noise.
  3. 3. A column in dollars and a column in years are clustered without scaling. What is likely?

    Larger numeric ranges dominate Euclidean distance, so standardize the features first.
  4. 4. What is soft clustering?

    Soft methods, such as Gaussian mixtures, assign each record a probability for every cluster.

Summary

  • Clustering groups similar records without labels; there is no single right answer.
  • Main families: partitioning, hierarchical, density-based, grid-based and model-based.
  • The distance measure and feature scaling shape the result as much as the algorithm does.
  • Hard clustering gives one cluster per record, soft clustering gives probabilities.

Next: k-Means and Its Variants