Skip to content
Tutorials Center
My learning

Topics

Lesson 31 of 40 ยท Machine Learning Forecasting

Regularized Linear Models for Forecasting

Forecast with Ridge, Lasso and Elastic Net on lag, calendar and Fourier features: scaling inside a pipeline, time-ordered tuning, sparse lag selection and why linear models extrapolate.

After trees, the simplest machine learning forecasters are linear models on lag features. They are fast, stable, easy to interpret and, unlike trees, they extrapolate: a linear trend keeps going. The only problem is that a rich feature table (many lags, many calendar dummies, Fourier terms) has many correlated columns, and ordinary least squares then overfits. Regularization fixes this by penalizing large coefficients.

  • Ridge (L2) shrinks all coefficients toward zero. Good when many features each carry a little signal.
  • Lasso (L1) shrinks some coefficients to exactly zero, so it also selects features, for example which lags matter.
  • Elastic Net mixes both, which is steadier when features are strongly correlated (and lags are).

The penalty strength is alpha. Features must be on a similar scale, so always put a StandardScaler in a Pipeline (see below for why a pipeline matters in time series).

A feature table with lags, calendar and a trend

Python Ridge on lags, weekday dummies and a time index

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The time index t is a feature a tree could never use well; here the linear model follows the rising trend naturally.

Lasso picks the lags that matter

Python Sparse lag selection with Lasso

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Lasso gives lags 1 and 7 (the real ones) large coefficients, keeps only tiny ones for lags 2 and 8 (their neighbours, which carry echoes of the same dynamics) and zeroes the other ten. This is a handy exploratory tool when you do not know which lags to include.

Tuning alpha without leaking the future

RidgeCV and LassoCV use random cross-validation by default, which mixes past and future. Pass a time-ordered splitter instead.

Python Choosing alpha with TimeSeriesSplit

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Each fold trains on an earlier block and validates on the next, as explained in time series cross-validation. Because the scaler sits inside the pipeline, it is re-fitted on each training fold and never sees validation data.

Fourier features for long seasonality

Daily data with a yearly cycle would need 365 dummies. A few sine and cosine pairs describe a smooth seasonal shape with far fewer columns (see trend and Fourier seasonality regression):

Python Fourier terms as regression features

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Because the Fourier columns and t are known for any future date, this model can forecast 90 days ahead directly, without feeding predictions back.

Strengths and limits

Strength Limit
Extrapolates trends Only linear effects unless you add interactions
Interpretable coefficients Needs scaling and careful alpha
Very fast, robust on short series Correlated lags make coefficients unstable (use Ridge/Elastic Net)
Works with Fourier and calendar features No automatic nonlinearity: combine with trees for residuals

A strong practical pattern is a linear model for trend and seasonality, then a tree model on the residuals. This gets the best of both: linear extrapolation and nonlinear interactions.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Which regularized model can set some coefficients exactly to zero?

    The L1 penalty of Lasso drives some coefficients to exactly zero, which acts as feature selection.
  2. 2. Why put StandardScaler inside the Pipeline when tuning with TimeSeriesSplit?

    Fitting the scaler on all data leaks validation statistics into training.
  3. 3. What advantage do linear models have over trees on a trending series?

    A linear model with a time or trend feature continues the trend, while a tree cannot predict outside its training range.
  4. 4. Why use Fourier terms instead of 365 daily dummies?

    Fourier terms give a compact, smooth seasonal representation, reducing overfitting.

Summary

  • Ridge, Lasso and Elastic Net tame many correlated lag and calendar features.
  • Lasso doubles as lag selection; Elastic Net is steadier with correlated lags.
  • Scale inside a Pipeline and tune with a time-ordered splitter.
  • Linear models extrapolate trends; Fourier terms handle long seasonal cycles with few columns.
  • Combine a linear base model with a tree model on its residuals for nonlinear effects.

Next: Neural Networks for Forecasting