Skip to content
Tutorials Center
My learning

Topics

Lesson 29 of 40 ยท Unsupervised Learning

Anomaly Detection and Manifold Learning

Find outliers with IsolationForest, LocalOutlierFactor and OneClassSVM, and explore nonlinear structure with t-SNE, Isomap and other scikit-learn manifold methods.

Two more unsupervised tasks complete the picture. Anomaly detection asks: which rows look unlike the rest? It powers fraud screening, fault detection and data cleaning. Manifold learning asks: can high-dimensional data be unrolled into a low-dimensional map that preserves who is near whom? It is mainly a visualization tool.

Outliers versus novelties

scikit-learn separates two situations:

  • Outlier detection: the training data already contains anomalies; you want to flag them. Use fit_predict.
  • Novelty detection: the training data is clean; you want to judge new points later. Use fit then predict.

Predictions are +1 for normal and -1 for anomalous.

Three detectors

IsolationForest Anomalies are easy to isolate with random splits.
LocalOutlierFactor Compares local density with neighbors.
OneClassSVM Draws a boundary around normal data.

All return +1 for normal and -1 for anomaly.

1 / 2

IsolationForest

It builds random trees that split the data on random features at random thresholds. Anomalies are few and different, so they get isolated in fewer splits. The average path length becomes the anomaly score. The key parameter is contamination, your estimate of the fraction of anomalies (it sets the cutoff).

Python IsolationForest

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

If you do not know the contamination, use contamination="auto" and choose a threshold on decision_function yourself, looking at the score distribution and the cost of false alarms.

LocalOutlierFactor

LOF compares each point's local density with that of its neighbors. A point in a sparse spot next to a dense region is an outlier even if its raw distance is modest. This catches anomalies that a global method misses when clusters have different densities.

Python LocalOutlierFactor and novelty mode

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

In novelty mode you may only call predict/decision_function on new data, not on the training set.

OneClassSVM

It learns a boundary around the normal data in a kernel space. It works best on clean, moderately sized training sets and is sensitive to nu (roughly the fraction of training errors allowed) and gamma. Scale features first.

Python OneClassSVM for novelty detection

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Evaluating detectors

True labels are rare in anomaly work. When you have a few, treat it as a ranking problem: use decision_function scores with ROC AUC or average precision (see evaluation metrics). Without labels, review the top-ranked rows by hand and ask domain experts.

Manifold learning

A manifold is a low-dimensional surface curled up in high-dimensional space, like a sheet of paper rolled into a swiss roll. Linear PCA cannot unroll it, while manifold methods try to preserve neighborhoods.

Method Preserves Notes
Isomap Distances along the surface (geodesics) Good for smooth manifolds
LocallyLinearEmbedding Local linear relationships Sensitive to noise
MDS Pairwise distances Classical embedding
SpectralEmbedding Neighbor graph structure Related to spectral clustering
TSNE Local neighborhoods Best for plots; not for new data
Python Unrolling a swiss roll with Isomap

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Isomap recovers the "unrolled" coordinate much better than PCA, whose straight axes cut across the layers.

t-SNE

t-SNE produces beautiful 2-D maps of high-dimensional data by keeping nearby points nearby. Rules for using it well:

  • It is for visualization only: distances between far clusters and cluster sizes are not meaningful.
  • The perplexity parameter (roughly the number of neighbors considered, 5-50) changes the picture; try a few values.
  • There is no transform for new data, only fit_transform.
  • Run PCA down to ~30-50 dimensions first for speed.
Python t-SNE map of the digits

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Digits of the same class form tight islands, a stronger separation than the two-component PCA plot showed. Remember that this is a picture for humans; do not feed t-SNE coordinates to a production model.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does a prediction of -1 mean for scikit-learn outlier detectors?

    Detectors use +1 for inliers and -1 for outliers.
  2. 2. Which detector compares a point's density with that of its neighbors?

    LOF's score is the ratio of neighbors' local density to the point's own; values far above 1 are anomalous.
  3. 3. What is the difference between outlier detection and novelty detection?

    In novelty mode you fit on clean data and call predict on unseen points.
  4. 4. Why should t-SNE output not be used as input features for a production model?

    t-SNE is a visualization tool: it cannot embed new points, and distances and cluster sizes are distorted.

Summary

  • Anomaly detectors return +1 (normal) or -1 (anomaly): IsolationForest is a strong default, LocalOutlierFactor handles varying density, OneClassSVM suits clean training data.
  • contamination sets how many points are flagged; scores let you choose your own cutoff.
  • Manifold learning (Isomap, LLE, MDS, TSNE) follows curved structure that PCA misses.
  • t-SNE is for plots only.

Next: Pipelines and ColumnTransformer