Skip to content
Tutorials Center
My learning

Topics

Lesson 21 of 76 ยท Cleaning and Preparing Data

Feature Engineering

Turn raw columns into useful features: ratios, date parts, bins, interactions, per-customer aggregates and log transforms, and avoid target leakage along the way.

A model, or a chart, can only use what the table shows it. Raw columns are often not the most informative way to say something. A column total_spent and a column n_orders are fine on their own, but their ratio, the average order value, may be the thing that separates big spenders from frequent small buyers. A timestamp is a single large number to a model, yet "day of week" and "hour" are things people can reason about.

Feature engineering is the craft of building new columns from existing ones so that the signal you care about is easy to see. It is where domain knowledge pays off most: a retail analyst knows that weekends behave differently, a lender knows that debt relative to income matters more than debt alone. Good features often beat a fancier algorithm.

This lesson covers the common recipes and one important danger, target leakage. It builds on the cleaning steps in Cleaning Text and Categories and prepares the ground for Encoding and Scaling Features.

flowchart LR
    R["Raw columns"] --> F["Feature engineering<br/>ratios, dates, bins, aggregates"]
    F --> E["Encoding and scaling"]
    E --> M["Model or analysis"]
    D["Domain knowledge"] --> F

Feature engineering sits between cleaned data and the model, guided by what you know about the problem.

Ratios and differences

A ratio puts two quantities on a common footing. Two customers with 20 orders each are very different if one spent 200 and the other spent 2,000. Useful ratios and differences include:

  • average order value = total spent / number of orders
  • debt-to-income = debt / income
  • price per square meter = price / area
  • change since last period = this month - last month

Always guard against division by zero. Pandas gives inf or NaN, which will confuse later steps, so decide explicitly what a zero denominator means (often: the ratio is missing, or zero).

Dates become several features

A single timestamp hides several patterns. The .dt accessor pulls them out: year, month, day of week, hour, whether it is a weekend, days since some reference date. The right parts depend on the question. A food-delivery company cares about hour and weekday; an insurer cares about the age of a policy.

Binning

Binning turns a number into ordered groups, such as age bands. Use pd.cut for fixed edges you choose, and pd.qcut for groups with equal counts (quantiles). Binning can make a relationship easier to explain and can tame outliers, but it throws away detail, so treat it as a choice, not a default.

Log transforms

Money, populations and counts are often right-skewed: most values are small, a few are huge. A logarithm compresses the long tail and turns multiplicative differences into additive ones, which many models and charts handle better. Use np.log1p(x), which computes log(1 + x) and is safe at zero.

Here is a small order table, with all four ideas applied together:

Python Ratios, dates, bins and logs

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Read the output column by column: price_per_item is a ratio, weekday, hour and is_weekend come from one timestamp, age_band uses edges you chose, and total_quartile puts about the same number of rows in each group.

Name features so their meaning is obvious: days_since_signup tells a reader far more than f17. Future you, and your teammates, will thank you.

Interactions

An interaction is a feature that combines two columns because their effect together differs from their separate effects. Advertising spend may only help when the product is in stock; temperature matters for ice-cream sales mainly on weekends. Simple ways to build one: multiply two numbers (area * rooms), combine two categories (region + "_" + plan), or compute a flag such as is_weekend & is_holiday.

Do not create every possible interaction. The number of pairs explodes, most are noise, and a flexible model such as a tree ensemble (Decision Trees, Random Forests and Boosting) finds many interactions on its own. Add the ones your domain knowledge suggests.

Aggregations per entity

Data is often stored at a finer level than the thing you want to predict. Orders are rows, but the question is about customers. The fix is to aggregate the fine-grained table per entity with groupby, producing one row per customer with features such as number of orders, total spend, average basket, days since the last order, and the share of orders on weekends.

For a window of time, say the last 30 days, filter first and then aggregate. The window must end before the moment you would use the prediction. That leads directly to the most important rule in feature work.

Target leakage

Target leakage means a feature contains information that would not be available at prediction time, usually because it is a consequence of the outcome. It produces models that look brilliant in testing and fail in real use.

Typical examples:

  • predicting whether a customer will churn using cancellation_reason (only filled in after they cancel);
  • predicting loan default using collections_calls (they happen because of default);
  • computing "orders per customer" over the whole history, including orders that come after the prediction date;
  • scaling or filling with statistics computed from the test rows too (a subtler form, covered in Train/Test Splits and Cross-Validation).

For every feature ask: at the moment of prediction, would I know this value? If not, remove it or compute it only from data before a cutoff date.

flowchart LR
    A["Past data<br/>build features here"] --> C(("Cutoff date"))
    C --> B["Future data<br/>outcome is observed here"]
    B -.->|"never use for features"| A

Features may only use information from before the cutoff; the outcome lives after it.

The next example builds customer features from an order log twice: once correctly with a cutoff, once leaking future orders. The target is "orders in the following 30 days". Watch how much better the leaky feature looks.

Python Per-customer features and a leakage check

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

n_orders_all correlates more strongly with the target than n_orders does (about 0.89 against 0.79), because it literally contains the answer; a model trained on it would collapse in production, where the future orders do not exist yet. The honest features, such as n_orders, show the weaker relationship, and that weaker number is the real one.

A practical checklist

  1. Write down what you will know at prediction time.
  2. Create ratios, date parts and aggregates that reflect how the domain works.
  3. Transform skewed positives with log1p; bin only when it helps explanation.
  4. Add interactions you can justify, not every pair.
  5. Check each feature for leakage; be suspicious of features that look too good.
  6. Keep feature code in a function so training and production compute them identically (see Pipelines and Hyperparameter Tuning).

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. Why is average order value (total spent / number of orders) often more informative than the two raw columns alone?

    A ratio separates "how much each order is worth" from "how often they order", which two raw totals mix together.
  2. 2. Which function creates groups with (roughly) equal numbers of rows?

    qcut cuts at quantiles, so each bin holds about the same count. pd.cut uses edges you choose, so counts can differ widely.
  3. 3. Why use np.log1p(x) instead of np.log(x) on a count column?

    log(0) is minus infinity, while log1p(0) is 0. It also compresses a long right tail.
  4. 4. Which feature is target leakage when predicting whether a customer will cancel next month?

    The reason is only recorded after the cancellation happens, so it would not be known at prediction time.
  5. 5. What is the safest way to build per-customer aggregates for a prediction?

    A cutoff guarantees the features contain only information that exists at prediction time.

Summary

  • Feature engineering builds new columns that make the signal easy to see; domain knowledge guides it.
  • Use ratios, date parts, bins (cut, qcut), interactions and log1p thoughtfully, and handle division by zero.
  • Aggregate fine-grained records per entity with groupby, using only data before a cutoff.
  • Target leakage, using information unavailable at prediction time, creates models that look great and fail in practice.
  • Ask of every feature: would I know this at the moment of prediction?

Next: Encoding and Scaling Features, turning categories and numbers into forms that models can use.