Skip to content
Tutorials Center
My learning

Topics

Lesson 6 of 40 ยท Data Preparation

Scaling and Normalization

Put features on comparable scales with StandardScaler, MinMaxScaler, RobustScaler, MaxAbsScaler and Normalizer, and learn which algorithms need scaling and which do not.

Real columns live on very different scales: age in tens, income in tens of thousands, a ratio between 0 and 1. Many algorithms compare or combine columns directly, so a column with large numbers can dominate the rest simply because its numbers are big. Feature scaling brings columns to comparable ranges. It is usually the first preprocessing step you will reach for.

Which models need scaling?

Need scaling Do not need scaling
k-nearest neighbors, SVM, k-means (distances) Decision trees
Linear and logistic regression with regularization Random forests
Neural networks, PCA Gradient boosting trees
Anything trained with gradient descent Naive Bayes (mostly)

Tree-based models split on thresholds one column at a time, so the scale of a column does not matter to them.

StandardScaler: mean 0, standard deviation 1

StandardScaler subtracts the column mean and divides by the standard deviation (a z-score). It is the default choice.

Python StandardScaler

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

After scaling, both columns have mean 0 and standard deviation 1, so a one-unit change means "one standard deviation" in either column. inverse_transform converts back.

MinMaxScaler: a fixed range

MinMaxScaler maps each column to a range, by default 0 to 1. It is useful when you need bounded values (images, neural networks) but is sensitive to outliers, because one extreme value squeezes everyone else.

Python MinMaxScaler and outliers

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The four normal values are crushed into the first 1% of the range by the outlier.

RobustScaler: outlier-resistant

RobustScaler uses the median and the interquartile range (IQR) instead of the mean and standard deviation, so extreme values hardly move the scale.

Python Compare three scalers on the same data

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

With RobustScaler the ordinary values stay spread out near zero while the outlier stays far away, which is usually what you want.

MaxAbsScaler and sparse data

MaxAbsScaler divides each column by its largest absolute value, giving the range [-1, 1] and keeping zeros as zeros. That makes it the right scaler for sparse matrices (such as text counts), where subtracting a mean would destroy sparsity.

Normalizer: scale rows, not columns

Every scaler above works per column. Normalizer works per row: it rescales each sample to unit length. This matters for similarity between samples, such as text vectors.

Python Normalizer works on rows

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Scaling changes the answer for distance-based models

Here is the practical payoff. k-nearest neighbors on the wine dataset: raw features include "proline" in the hundreds, which swamps columns near 1.

Python Scaling improves k-NN

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Notice the scaler is fitted on the training set only, then applied to both. Pipelines, covered later in the course, bundle this into one object so you cannot forget.

Choosing a scaler

  • Not sure? StandardScaler.
  • Need a bounded range and the data has no big outliers? MinMaxScaler.
  • Outliers present? RobustScaler.
  • Sparse data? MaxAbsScaler.
  • Comparing whole samples (text, angles)? Normalizer.

Do not scale the target y for classification. For regression, scaling y is sometimes useful but must be undone before reporting predictions (see TransformedTargetRegressor).

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Which model is LEAST affected by feature scaling?

    Trees split on thresholds per column, so monotonic rescaling does not change their splits.
  2. 2. Which scaler is the best choice when the data has strong outliers?

    RobustScaler uses the median and IQR, which barely move with extreme values.
  3. 3. On which data should the scaler's fit be called?

    Fit on training data, then transform both train and test, to avoid leakage.
  4. 4. What does Normalizer scale?

    Normalizer works per sample (row), unlike the column scalers.

Summary

  • Scaling puts features on comparable ranges; distance-based, regularized and gradient-trained models need it, tree models do not.
  • StandardScaler (default), MinMaxScaler (bounded), RobustScaler (outliers), MaxAbsScaler (sparse), Normalizer (rows).
  • Always fit scalers on training data only.
  • Scaling can noticeably lift k-NN and SVM scores.

Next: Handling Missing Values