Skip to content
Tutorials Center
My learning

Topics

Lesson 28 of 40 ยท Machine Learning Forecasting

Lag Features and Supervised Framing

Turn a time series into a table that scikit-learn can learn from: lag columns, rolling statistics, calendar features, and a leakage-safe chronological train/test split.

Statistical models such as ARIMA are built for one series at a time. Machine-learning models (linear regression, random forests, gradient boosting, neural networks) expect a table: rows of examples, columns of features, and a target. To use them for forecasting, you must reshape the series into such a table. This is called supervised framing or a "windowed" dataset, and it is the foundation of every ML forecaster.

The idea is simple: to predict y[t], the model may look at what was known before t. So the features are built from past values (lags), summaries of the recent past (rolling statistics), and calendar information that is known in advance.

Lag features

A lag-k feature at row t is the value y[t-k]. In pandas it is one call: shift(k).

Python Build lag features with shift

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The first three rows contain NaN because the lags reach before the start of the data. You drop those rows (dropna()), which costs one row per lag of the largest lag you use.

flowchart LR
    subgraph window["Features at row t"]
        L1["y[t-1]"]
        L2["y[t-2]"]
        L7["y[t-7]"]
        R["rolling mean of the last 7"]
        C["day of week, month"]
    end
    window --> M["ML model"]
    M --> P["Prediction of y[t]"]

Every feature is computed only from information that was available before time t.

Rolling and expanding statistics

Raw lags show individual past values. Rolling statistics summarize a whole window: the recent level (mean), the recent volatility (standard deviation), the recent range. The crucial detail is to shift by one after computing the rolling window, otherwise the window includes the current value, which is the very thing you are trying to predict.

Python Rolling features without leakage

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Check row 4: roll3_wrong averages y[t-2], y[t-1], y[t] (it already "knows" the answer), while roll3_ok averages the three previous values only. A model trained with the wrong column looks superb in testing and collapses in production. That is data leakage, and it is the number-one bug in time series ML.

Calendar features

Calendar properties are known far in advance, so they are safe features and often powerful ones.

Python Calendar columns from the index

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Tree models can use dayofweek as a number directly. A linear model cannot: Monday = 0 and Sunday = 6 are neighbors in time but far apart as numbers, so use dummy variables or sine/cosine pairs (see the Fourier lesson).

Putting it together and splitting in time

Build the table once, then split by time, never by shuffling. The test set must lie entirely after the training set. Shuffled splitting lets the model peek at the future's neighbors and inflates scores.

Python A leakage-safe table and chronological split

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Always print a naive baseline next to your model. Here "predict yesterday's value" is the baseline; a model that cannot beat it is not worth deploying. The seasonal naive baseline (predict the value from 7 days ago) would be even harder to beat for this weekly series.

A handy sanity check: if your ML model beats the naive baseline by a huge margin on a noisy real-world series, suspect leakage before celebrating.

The one-step view and its limits

The table above predicts one step ahead using actual past values. If you need a forecast several days out, the lags of the future rows are not yet observed. Handling that honestly is the subject of multi-step forecasting strategies.

Choosing which lags to include

  • Use lags suggested by the ACF/PACF (see the ARIMA lessons).
  • Always include the seasonal lag (7 for weekly, 12 for monthly, 24 for hourly data).
  • Add rolling means over a few windows (7, 30) for level; a rolling standard deviation for volatility.
  • Do not add dozens of lags blindly: highly collinear columns help trees little and hurt linear models.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does df['y'].shift(3) produce in row t?

    shift(k) moves values down by k rows, so row t receives y[t-k].
  2. 2. Why do we write y.shift(1).rolling(7).mean() instead of y.rolling(7).mean()?

    Without the shift, the window includes y[t], the target itself, so the feature leaks the answer.
  3. 3. How should a time series be split into training and test data?

    Forecasting predicts the future, so evaluation must use only later data than the training data; shuffling leaks future information.
  4. 4. Why is a naive baseline important?

    If a complex model does not beat a trivial forecast like yesterday's value, it adds no value.

Summary

  • ML forecasting reshapes a series into a table: lags, rolling statistics and calendar features as inputs, the next value as the target.
  • Compute rolling features on y.shift(1) so the window never includes the target.
  • Split chronologically; never shuffle. Compare against a naive baseline.
  • A one-step table is only the start: multi-step forecasts need a strategy.

Next: Tree Models for Forecasting