Skip to content
Tutorials Center
My learning

Topics

Lesson 8 of 40 ยท Data Preparation

Encoding Categorical Features

Turn text categories into numbers with OneHotEncoder, OrdinalEncoder, LabelEncoder and TargetEncoder; handle unseen categories, rare values and high-cardinality columns.

Models work with numbers, but real data is full of categories: city, colour, plan type, country. Encoding converts them into numbers without inventing false meaning. The right encoder depends on whether the categories have an order and how many there are.

Why not just number them?

If you replace Oslo, Rome and Paris with 0, 1 and 2, a linear model concludes that Paris is "twice Rome" and that Rome sits between the others. For categories with no natural order, that is nonsense. One-hot encoding avoids it.

OneHotEncoder

One-hot encoding creates one 0/1 column per category. Exactly one of them is 1 in each row.

Python OneHotEncoder

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

sparse_output=False gives a normal array so you can read it. For wide data, the default sparse matrix saves memory.

Dropping a column to avoid redundancy

If you know a row is not Oslo and not Rome, it must be Paris, so one column is redundant. For linear models that creates perfect collinearity. drop="first" removes one column per feature. For binary features use drop="if_binary".

Python drop and unseen categories

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

handle_unknown="ignore" encodes a never-seen category as all zeros instead of crashing at prediction time. This is almost always what you want in production.

Rare categories and high cardinality

A column like zip_code may have thousands of values, and one-hot encoding would create thousands of columns. Two built-in options:

  • min_frequency=10 groups categories that appear fewer than 10 times into an "infrequent" bucket.
  • max_categories=20 keeps the 20 most common and lumps the rest together.
Python Group rare categories

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

OrdinalEncoder: when order exists

For categories with a real order (small, medium, large; low, mid, high), OrdinalEncoder assigns integers following the order you specify.

Python OrdinalEncoder with explicit order

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Always pass categories= for ordered data. OrdinalEncoder is also fine for tree-based models on unordered categories, because trees can split an integer code into groups, though one-hot is safer for linear models.

LabelEncoder is for the target only

LabelEncoder turns class names in y into integers 0..n-1. It is not meant for feature columns. (Most classifiers accept text labels directly, so you often do not need it.)

Python LabelEncoder for labels

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

TargetEncoder: replace a category with its average outcome

TargetEncoder (added in version 1.3) replaces each category by a smoothed average of the target for that category. It keeps one column however many categories there are, so it suits high-cardinality features. It uses internal cross-fitting in fit_transform to limit leakage, and it must be fit on training data only.

Python TargetEncoder

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Encoding DataFrame columns

Real tables mix numbers and categories, and each needs different handling. That is the job of ColumnTransformer, covered in the pipelines chapter.

Python pandas get_dummies versus OneHotEncoder

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

pd.get_dummies is fine for exploration, but it does not remember the categories, so train and test can end up with different columns. OneHotEncoder learns the categories once in fit and applies them consistently, which is why it is preferred in models.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. You encode city names as 0, 1, 2 for a linear model. What goes wrong?

    Integer codes imply an order and distances that unordered categories do not have. Use one-hot instead.
  2. 2. Which setting prevents a crash when prediction data has a category never seen in training?

    handle_unknown='ignore' encodes the new category as all zeros.
  3. 3. Which encoder suits sizes small < medium < large?

    OrdinalEncoder keeps the order, provided you pass categories explicitly.
  4. 4. What is LabelEncoder intended for?

    LabelEncoder is for target labels; use OrdinalEncoder or OneHotEncoder for features.
  5. 5. Why use TargetEncoder for a column with 5,000 categories?

    It keeps dimensionality low by encoding each category as a smoothed target mean.

Summary

  • Use OneHotEncoder for unordered categories, with handle_unknown="ignore" for safety.
  • Use OrdinalEncoder(categories=[...]) for ordered categories.
  • LabelEncoder is only for y.
  • Group rare values with min_frequency or max_categories; use TargetEncoder for very high cardinality.
  • Prefer scikit-learn encoders over get_dummies inside models.

Next: Binning, Power Transforms and Custom Features