Skip to content
Tutorials Center
My learning

Topics

Lesson 9 of 39 ยท Data Preprocessing

Similarity and Distance Measures

Learn Euclidean, Manhattan, Minkowski, cosine, Jaccard, Hamming and Gower measures, when to use each, and how to compute them in Python with scipy and NumPy.

Clustering, nearest-neighbor classification, outlier detection and recommendation all rest on one question: how alike are two records? A distance (dissimilarity) is small for similar records; a similarity is large for them. The answer depends on the attribute types, so there is no single best measure. This lesson covers the standard ones and a rule for choosing.

Properties of a distance

A true metric d(x, y) satisfies: it is never negative, d(x, y) = 0 only if x = y, it is symmetric (d(x, y) = d(y, x)) and obeys the triangle inequality (d(x, z) <= d(x, y) + d(y, z)). Euclidean, Manhattan and Hamming are metrics. Cosine distance is not quite, which rarely matters in practice.

Numeric attributes

The Minkowski distance is a family: d = (sum |xi - yi|^p)^(1/p).

  • p = 2: Euclidean, the straight-line distance.
  • p = 1: Manhattan (city block), the sum of absolute differences; less sensitive to one large difference.
  • p = infinity: Chebyshev, the largest single difference.
Python Minkowski family

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Because these distances add raw differences, scale first (Data Transformation). Also remember the curse of dimensionality: in hundreds of dimensions, nearest and farthest neighbors become almost equally far.

Python Why scaling matters for distance

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Unscaled, person 1 (26, 90k) looks far from person 0 and person 2 (60, 51k) looks close: age is nearly ignored. After scaling, both differences count.

Cosine similarity

For text and other high-dimensional sparse vectors, direction matters more than length. Cosine similarity is (x . y) / (|x| |y|), from -1 to 1 (0 to 1 for non-negative data). Two documents with the same word proportions but different lengths are cosine-identical.

Python Cosine similarity of word counts

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Binary and set attributes

For baskets and yes/no attributes, count matches. Let q = attributes where both are 1, r and s where exactly one is 1, t where both are 0.

  • Simple matching coefficient: (q + t) / (q + r + s + t). Right for symmetric binary attributes (gender).
  • Jaccard similarity: q / (q + r + s). Ignores shared zeros, so it suits asymmetric attributes such as "bought the item". Two customers who both did not buy 9,990 of 10,000 items are not alike for that reason.
  • Hamming distance: the number of positions that differ.
Python Simple matching versus Jaccard

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Nominal and mixed attributes

For nominal attributes with many values, use the share of mismatching attributes. Real tables mix types, so Gower's distance averages per-attribute dissimilarities: range-scaled absolute difference for numbers, 0 or 1 for categories. It copes with missing values by skipping them.

Python A small Gower distance

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Correlation and other measures

  • Pearson correlation as a similarity between two profiles (such as two users' ratings) ignores each person's habit of rating high or low.
  • Mahalanobis distance accounts for the spread and correlations among attributes. It underlies outlier detection and is used again in the anomaly chapter.
  • Edit (Levenshtein) distance compares strings; dynamic time warping compares time series that are shifted in time.
Python Distance matrix for a small data set

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Choosing a measure

Data Good choice
Numeric, scaled Euclidean or Manhattan
Numeric with outliers or many dimensions Manhattan
Sparse text vectors Cosine
Sets and asymmetric binary Jaccard
Symmetric binary / nominal Simple matching, Hamming
Mixed types Gower

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Euclidean distance between (0, 0) and (3, 4) is:

    sqrt(3 squared + 4 squared) = 5. Seven is the Manhattan distance.
  2. 2. Which similarity ignores attributes where both records have 0?

    Jaccard counts only attributes where at least one record has a 1.
  3. 3. Which measure compares the direction of vectors rather than their length?

    Cosine similarity depends on the angle between vectors, so document length does not matter.
  4. 4. What must you usually do before using Euclidean distance on columns with different units?

    Otherwise columns with large numeric ranges dominate the distance.
  5. 5. Which measure suits a table mixing numbers and categories?

    Gower averages per-attribute dissimilarities of different types.

Summary

  • A distance is small for similar records; a metric satisfies non-negativity, symmetry and the triangle inequality.
  • Numeric: Minkowski (Manhattan, Euclidean, Chebyshev); scale first.
  • Sparse text: cosine. Sets and asymmetric binary: Jaccard. Mixed: Gower.
  • The curse of dimensionality weakens distances in many dimensions.
  • The choice of measure often changes results more than the choice of algorithm.

Next: Building a Preprocessing Pipeline