Skip to content
Tutorials Center
My learning

Topics

Lesson 74 of 111 ยท Beyond the Basics

Machine Learning for Time Series

Turn forecasting into regression: lag and rolling features, seasonality features, naive and seasonal-naive baselines, and walk-forward evaluation with scikit-learn.

A time series is data in order: daily sales, hourly electricity use, weekly sign-ups. The task is usually forecasting: given what happened up to today, predict what happens next. You do not need special software to start. If you describe each day by what was true before it (yesterday's value, last week's value, the average of the last seven days, the day of the week), forecasting becomes an ordinary regression problem that the models from earlier lessons can solve.

Two things make time series different. Rows are not interchangeable, so you cannot shuffle them, and the easiest way to cheat is to let tomorrow's information leak into today's features. This lesson builds features carefully, sets honest baselines, and evaluates by walking forward through time.

Forecasting as regression

Lag features Values from 1, 7, 28 days ago.
Rolling stats Recent mean and spread.
Calendar Day of week, month, holidays.
Target Tomorrow's value.
1 / 2

A series to practice on

We generate 400 days of sales with three ingredients: a slow upward trend, a weekly pattern (weekends are busier) and random noise. A fixed seed makes every run identical.

Baselines first

Before any model, forecast with rules that need no learning. As in Baselines and Dummy Models, they tell you how much a model has to earn.

  • Naive forecast: tomorrow equals today.
  • Seasonal naive: tomorrow equals the same weekday last week (for a weekly cycle, a lag of 7).

On series with strong seasonality, seasonal naive is hard to beat. If your model cannot do better, it is not helping.

Python Naive and seasonal-naive baselines

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The training mean ignores trend and weekdays, so it does poorly; naive copies yesterday and gets the weekday pattern wrong; seasonal naive is much better because it respects the weekly cycle. This is the bar a machine learning model must clear.

Lag and rolling features

For each day we build features from the past only:

  • Lags: sales.shift(1), shift(7), shift(14).
  • Rolling statistics: mean or standard deviation over the last 7 days. Careful: rolling(7).mean() at day t includes day t itself, which is the value we want to predict. Always shift(1) first, so the window covers days t-7 to t-1.
  • Calendar features: day of week (one-hot, since "Sunday" is not 6 times "Monday") and month. A cyclic encoding with sine and cosine works for features that wrap around, as in Feature Creation.
  • Trend: a simple day counter, or rely on lags and the rolling mean to carry it.

The first rows have no history, so lags and rolling windows produce NaN. Drop those rows; do not fill them with future-looking values.

Python Features and a regression forecast

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Ridge, which uses the weekday dummies and recent levels, beats seasonal naive (MAE about 3.3 versus 4.2). Gradient boosting does worse (about 5.7), and the reason is instructive: trees cannot extrapolate beyond the values they saw in training, so on a series that keeps trending upward a forest or boosting model falls behind; linear models or tree models fitted on differences (today minus yesterday, or minus last week) handle trends better. The leaky row shows the danger: including today in the rolling mean lowers the error to about 2.9, a score the model could never reach in production, where today's value is unknown. Here the leak is mild because the noise is small; on real data it is often dramatic.

A time series model that scores suspiciously well almost always has leakage. Check that every feature at time t is computable using only data from before t, and that the same holds at prediction time.

Walk-forward evaluation

One train/test split gives one score, and that score depends on which weeks you happened to hold out. Walk-forward (rolling-origin) evaluation repeats the test many times: train on days up to a cutoff, forecast the next block, move the cutoff forward, and average the errors. Every test block lies after its training data, just like the real future. scikit-learn's TimeSeriesSplit does exactly this with an expanding training window; see Cross-Validation for Groups and Time for more.

In real use you often need a horizon beyond one day. Two strategies: a recursive model predicts one day ahead and feeds its predictions back as lags, so errors accumulate; a direct approach trains a separate model per horizon (target y shifted by 1, 7, 14 days). Direct models avoid error build-up and are easy to write with the same features.

Python Walk-forward evaluation and a 7-day-ahead model

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

For horizon 7, today's value is the same weekday as the target day, so seasonal naive uses lag_0; for horizon 1 the same weekday last week is 6 days before today (lag_6). The per-fold MAE values vary (roughly 3.1 to 4.2 for horizon 1), which is why a single split can mislead: report the mean and the spread. Ridge beats seasonal naive at both horizons, with a mean MAE of about 3.5 versus 4.5.

Seasonality, decomposition and SARIMA

Lag features are not the only way to forecast. The classical statistical tools describe the structure of a series directly, and they are worth knowing because they explain why lag features work, and they make strong baselines. This section uses statsmodels, which loads in the browser runner on first use (the first run is slower).

Decomposition with STL

Most business series are a sum of three parts: a trend (the slow level), a seasonal pattern that repeats with a fixed period (7 for weekly data on daily readings) and a remainder (everything else). STL (seasonal-trend decomposition using LOESS) separates them. Plotting the three parts tells you quickly whether a series has a trend worth modeling, how strong the seasonality is, and whether the remainder looks like noise. Prophet-style forecasters are built on the same additive idea: trend plus seasonal terms plus holiday effects.

Reading ACF and PACF

The autocorrelation function (ACF) gives the correlation between the series and itself shifted by k steps. A peak at lag 7 on daily data means "this day resembles the same weekday last week". The partial autocorrelation function (PACF) is the correlation at lag k after removing the effect of the shorter lags, so it shows the direct link. They are the tools for choosing the orders of an ARIMA model.

Many models need a stationary series, one whose mean and spread do not drift. You get there by differencing: subtracting the previous value removes a trend, and subtracting the value one season back (diff(7)) removes a repeating weekly pattern. Rolling statistics (a moving mean and standard deviation) are the quick visual check: if the rolling mean keeps climbing, the series is not stationary.

Python STL decomposition, ACF and differencing

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

STL finds the weekly pattern we planted: Monday to Thursday sit below the weekly average, Saturday and Sunday about 13 to 16 above it. The trend runs from about 110 to 170, which is our 100 to 160 line plus the average weekday effect, and the remainder has a standard deviation of 3.4, close to the true noise of 4. The ACF of the raw series is large at every lag (0.84 at lag 1, 0.91 at lag 7), the signature of a trend plus a cycle. After one ordinary difference the lag-1 value is near zero (-0.08), but lag 7 is still 0.71, so the weekly cycle remains. Differencing once more at lag 7 removes the big cycle, and what remains is a negative correlation of about -0.5 at lags 1 and 7. That pattern is the typical sign that a moving-average term at lag 1 and one at lag 7 are needed, which is what the seasonal model below includes.

SARIMA

ARIMA(p, d, q) combines three ingredients: p autoregressive terms (the series depends on its own recent values), d rounds of differencing, and q moving-average terms (the series depends on recent forecast errors). SARIMA(p, d, q)(P, D, Q, m) adds the same three ingredients at the seasonal lag m. For our daily series with a weekly cycle, m = 7, and a good starting point is (1, 1, 1)(1, 1, 1, 7): one ordinary and one seasonal difference, with one AR and one MA term each. Use the ACF and PACF to adjust, and compare candidate models by AIC (lower is better) and by error on held-out data, not by in-sample fit.

The example holds out the final 28 days, fits a plain ARIMA and a SARIMA on the earlier part, and compares both against the seasonal-naive baseline from earlier in this lesson.

Python ARIMA versus SARIMA against a seasonal-naive baseline

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Plain ARIMA ignores the weekly cycle and forecasts a smooth line, so its MAE (about 7.5) is worse than the seasonal-naive baseline (4.0). Adding the seasonal part cuts the error to about 3.0, below the baseline, and its 95% prediction interval covers all 28 test days. A forecast with an interval is a real advantage of this family: you get a range, not only a number. Notice also how SARIMA reproduces the weekend peak in the first week of forecasts (177.7 and 174.7 against actual 182.3 and 177.6).

Choose the tool by the problem. SARIMA is excellent for one series with a stable seasonal period and little extra data. Lag-feature models with gradient boosting scale better to thousands of series and extra inputs such as prices or promotions. Always keep the seasonal-naive baseline in the comparison.

Things to remember

  • Do not shuffle and do not use plain k-fold; use time-ordered splits.
  • Seasonality matters: weekly, yearly, holiday and promotion effects often explain more than the model choice.
  • Non-stationarity: trends and shifts mean the future may differ from the past. Refit regularly and monitor error.
  • Missing and irregular timestamps should be handled before building lags (resample to a fixed frequency first).
  • For purely statistical alternatives (ARIMA, exponential smoothing), see the statsmodels library; for hard problems, deep sequence models exist, but a feature-based gradient boosting model with strong baselines wins surprisingly often.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why must you call shift(1) before computing a rolling mean for a 'predict today' feature?

    A rolling mean at day t normally includes day t. Shifting by one makes the window end at t-1, using only information available before the prediction.
  2. 2. What is a seasonal-naive forecast for daily data with a weekly cycle?

    It repeats the value seen one season (7 days) ago, a strong baseline whenever the weekly pattern dominates.
  3. 3. Why is TimeSeriesSplit preferable to shuffled k-fold for forecasting?

    Shuffled folds train on the future to predict the past, which inflates scores. Time-ordered splits never do.
  4. 4. Why can tree-based models struggle on a strongly trending series?

    A tree outputs averages of training targets. Modelling differences or adding a linear trend component fixes this.

Summary

  • Forecasting becomes regression when you describe each time step by features from before it.
  • Lags, shifted rolling statistics and calendar features are the staple inputs.
  • Always compare with naive and seasonal-naive baselines.
  • Prevent leakage: shift rolling windows and use only information available at forecast time.
  • Evaluate with walk-forward splits, and consider direct models per horizon.
  • Expect the series to drift; refit and monitor.

Next, learning from mostly unlabelled data: Semi-Supervised Learning and Active Learning.