Skip to content
Tutorials Center
My learning

Topics

Lesson 69 of 111 ยท Unsupervised Learning

t-SNE and Manifold Learning

Visualize high-dimensional data with t-SNE and Isomap, learn what perplexity does, why t-SNE plots mislead, and how to check an embedding with trustworthiness.

PCA can only rotate and flatten. If your data lies on a curved sheet, like a rolled-up carpet, a straight projection squashes layers on top of each other. Manifold learning methods assume the data lives on a low-dimensional curved surface (a manifold) inside the high-dimensional space, and try to unroll it while keeping neighbours close.

The best-known tool is t-SNE, which turns 64-pixel digit images into a 2D map where each digit forms its own island. It is great for looking at data and treacherous for concluding things from it. This lesson shows how to use it, how to check it, and where it lies.

Manifold learning in one idea

Unroll Data lies on a curved sheet in many dimensions.
Keep neighbours Points close in the original space stay close.
Draw a map Usually 2D, for human eyes.
1 / 2

Isomap: distances along the surface

Take a "Swiss roll": a flat sheet rolled up like a pastry. Two points on neighbouring layers are close in a straight line but far apart if you must walk along the sheet. Isomap builds a graph that connects each point to its n_neighbors nearest points, measures shortest paths along that graph (approximating distances along the surface), and then finds a low-dimensional layout that reproduces those distances.

Python Unrolling a Swiss roll

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Colors show the position along the roll. In the PCA panel the colors overlap because the roll is flattened like a pancake. In the Isomap panel they form a smooth gradient: the sheet has been unrolled. Isomap is sensitive to n_neighbors: too small and the graph falls apart into pieces, too large and it creates shortcuts across layers that ruin the unrolling.

t-SNE: neighbourhoods, not distances

t-SNE (t-distributed Stochastic Neighbor Embedding) works differently. For each point it computes the probability that another point would be picked as its neighbour, using a Gaussian bump in the original space. It then places points in 2D and moves them until the same kind of neighbour probabilities, computed with a heavy-tailed t-distribution, match. Points that were neighbours get pulled together; others are pushed apart. The heavy tails are what let clusters separate into distinct islands.

The key setting is perplexity, roughly "how many neighbours each point pays attention to". Typical values are 5 to 50. Low perplexity focuses on very local structure and can shatter real clusters; high perplexity blurs toward global structure.

In scikit-learn, TSNE has fit_transform but no transform: you cannot embed new points into an existing map, so it is only for exploration, never as a preprocessing step in a pipeline.

Python Digits on a t-SNE map

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

PCA smears the ten digits into overlapping clouds, with a few easy ones such as 0 pulled apart. t-SNE shows ten fairly clean islands. The final kl_divergence_ is the quantity t-SNE minimizes; lower means a better match, but it cannot be compared across different perplexities.

Pass init="pca" (the default in current scikit-learn) and a fixed random_state for reproducible, more stable layouts. On large data, reduce to about 30โ€“50 dimensions with PCA first; it removes noise and makes t-SNE much faster.

Why t-SNE plots mislead

A t-SNE picture feels like a map, but only some of its features are real.

  • Cluster size means nothing. t-SNE adapts to local density, so a tight cluster and a wide one can look the same size.
  • Distances between clusters mean little. Two islands drawn far apart may not be more different than two drawn close together. Only the fact that points are neighbours is trustworthy.
  • Perplexity changes the story. The same data can look like three clusters or ten depending on this setting. Always try several values.
  • Random data can show structure. At low perplexity, pure noise can appear as clumps.
  • Different seeds, different pictures. Rotations and flips are meaningless; real structure should survive a change of seed.
Python Same data, three perplexities, plus a trust score

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The blob with cluster_std=2.5 is five times wider than the first, yet t-SNE adapts to local density, so on the map the widths of the clusters are far less different than in the data. Do not read spread off the picture. Trustworthiness (from sklearn.manifold) is a useful sanity check: it measures, on a scale up to 1, whether points that are neighbours in the embedding were also neighbours in the original space. Values near 1 mean few false neighbours.

Other methods and UMAP

MDS preserves pairwise distances (slow, mostly for small data), LocallyLinearEmbedding rebuilds each point from its neighbours, and SpectralEmbedding uses a graph Laplacian. UMAP is a popular newer method, usually faster than t-SNE and better at keeping some global structure, and unlike t-SNE it can transform new points. It is a separate package that is not part of scikit-learn, so this is a read-only example:

Python
# pip install umap-learn   (not available in the browser runner)
import umap

reducer = umap.UMAP(n_neighbors=15, min_dist=0.1, random_state=0)
embedding = reducer.fit_transform(X)      # shape (n_samples, 2)
new_points = reducer.transform(X_new)     # works for unseen data

UMAP shares t-SNE's cautions: the picture depends on its settings, and cluster sizes and gaps should not be over-read.

A sensible workflow

  1. Standardize features and, for wide data, reduce with PCA to a few dozen dimensions.
  2. Embed with t-SNE at two or three perplexities and compare.
  3. Colour the points by something you know (a label, a category) to see if the map is meaningful.
  4. Check trustworthiness, then use clustering such as DBSCAN on the original features, not on the 2D map.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why can Isomap unroll a Swiss roll when PCA cannot?

    Isomap approximates geodesic (along-the-surface) distances through a nearest-neighbour graph, so layers that are close in a straight line but far along the sheet stay apart.
  2. 2. In a t-SNE plot, what can you trust the most?

    t-SNE preserves local neighbourhoods. Cluster sizes, gaps between clusters and axis directions are artefacts of the optimization.
  3. 3. What does perplexity control in t-SNE?

    Perplexity is an effective neighbourhood size. Very low values fragment clusters; very high values blur local detail.
  4. 4. Why is TSNE unsuitable as a step in a prediction pipeline?

    scikit-learn's TSNE only offers fit_transform. New data cannot be placed on an existing map, so it is for visualization only.

Summary

  • Manifold learning flattens curved, low-dimensional structure that linear PCA cannot.
  • Isomap preserves distances along the surface; tune n_neighbors.
  • t-SNE preserves neighbourhoods and forms islands; tune perplexity, set random_state, and try several values.
  • Do not read meaning into cluster size, spacing or orientation.
  • trustworthiness checks whether neighbours survive the embedding.
  • UMAP is a faster alternative outside scikit-learn; treat its plots with the same caution.

Next: more ways to shrink data, in Other Dimensionality Reduction.