Skip to content
Tutorials Center
My learning

Topics

Lesson 42 of 45 ยท Projects

Project: Forecasting a Time Series

Forecast a time series: build sliding windows, set honest baselines, split by time, and compare a PyTorch LSTM (read-only) with a live autoregressive model.

Forecasting predicts future values of something measured over time: sales, temperature, server load, electricity use. It looks like an ordinary regression problem, and that is the trap. The data have an order, the future must never leak into the past, and the best predictor is often embarrassingly simple. In this project you will turn a series into training examples with sliding windows, set baselines that any real model has to beat, split the data by time rather than at random, and forecast several steps ahead.

PyTorch cannot run on this site, so the LSTM forecaster is read-only. The live editors use a synthetic series (a slow trend, two seasonal cycles and noise) so that we know what the right answer looks like, and an autoregressive model: a linear regression (scikit-learn's Ridge) on the previous 24 values. It uses the same windows as the LSTM would, so everything except the model itself transfers directly. Read RNNs, LSTMs and GRUs first if recurrent layers are new.

The forecasting workflow

Windows Past values in, next value out.
Baselines Last value, same time last cycle.
Time split Train on the past, test on the future.
Forecast Predict several steps ahead.
1 / 2

Turning a series into examples

A supervised model needs pairs of inputs and targets. A time series offers them through windowing: choose a window length L, take L consecutive values as the input, and use the very next value as the target; then slide the window forward by one step. A series of length n produces n - L examples. A window of 4 over the values v0, v1, v2, ... gives (v0..v3) -> v4, (v1..v4) -> v5, and so on. In PyTorch, the inputs become a tensor of shape (examples, L) for a linear model or (examples, L, 1) for an LSTM (batch, time, features), and you hand them to a DataLoader. Split into past and future first; only then is it fine to shuffle the training windows into mini-batches.

Baselines first

Two forecasts need no learning at all. The last-value forecast says "tomorrow equals today". The seasonal naive forecast says "this hour equals the same hour one cycle ago". Any model you build must beat them clearly, otherwise it has learned nothing useful. The first editor builds the series, shows what windows look like, and measures both baselines on the last 60 points (a chronological test set). The metric is the mean absolute error (MAE), in the units of the series.

Python The series, windows and baselines

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The series has 300 points: the first 240 are for training and the last 60 are the test period. The first example from the 4-value window is [0.05, 1.79, 2.86, 2.25] -> 1.46. The test part has a standard deviation of 2.37, so an error of 2.4 would mean "no better than predicting the average". The last-value baseline has an MAE of 1.013 and the seasonal naive baseline 0.829. Both are decent, because the series is smooth, but both are limited: last value cannot follow a cycle, and seasonal naive ignores the upward trend (each cycle ends higher than the previous one) as well as the noise in the old value.

The LSTM forecaster in PyTorch

This is the neural version, read-only. The model reads a window and predicts the next value from the last hidden state:

Python
import torch
from torch import nn
from torch.utils.data import DataLoader, TensorDataset

window = 24
z = torch.tensor((series - series[:240].mean()) / series[:240].std(), dtype=torch.float32)

def windows(z, L):
    X = torch.stack([z[i:i + L] for i in range(len(z) - L)]).unsqueeze(-1)   # (N, L, 1)
    y = z[L:]
    return X, y

X, y = windows(z, window)
is_test = torch.arange(len(y)) + window >= 240          # split by time, not at random
train_loader = DataLoader(TensorDataset(X[~is_test], y[~is_test]), batch_size=32, shuffle=True)

class Forecaster(nn.Module):
    def __init__(self, hidden=32):
        super().__init__()
        self.lstm = nn.LSTM(input_size=1, hidden_size=hidden, num_layers=1, batch_first=True)
        self.head = nn.Linear(hidden, 1)

    def forward(self, x):                                # x: (batch, time, 1)
        out, (h, c) = self.lstm(x)                       # out: (batch, time, hidden)
        return self.head(out[:, -1]).squeeze(-1)         # predict from the last time step

model = Forecaster()
optimizer = torch.optim.Adam(model.parameters(), lr=1e-2)
loss_fn = nn.MSELoss()
for epoch in range(100):
    for xb, yb in train_loader:
        optimizer.zero_grad()
        loss_fn(model(xb), yb).backward()
        optimizer.step()

@torch.no_grad()
def forecast(model, history, steps):
    model.eval()
    hist = list(history)
    for _ in range(steps):
        x = torch.tensor(hist[-window:], dtype=torch.float32).view(1, window, 1)
        hist.append(model(x).item())                     # feed the prediction back in
    return hist[len(history):]

Three details. The series is normalized with the training mean and standard deviation only. batch_first=True makes the LSTM expect (batch, time, features). And forecast is recursive: for each new step it appends its own prediction to the history, because at forecast time the real future values are not known.

For a clean, strongly periodic series like ours, an LSTM is not obviously better than the linear model below, and it is slower to train and harder to tune. That is not a criticism of LSTMs; it is the reason you try the simple model first. A neural network starts paying off with several input features, nonlinear relationships and large amounts of data.

The autoregressive stand-in

An autoregressive model predicts the next value as a weighted sum of the previous L values, which is linear regression on the windows. With L = 24 it can represent the 24-step seasonal cycle, the 6-step cycle and the trend, all at once. The next editor does four things:

  1. Normalizes with training statistics and builds the windows.
  2. Splits by time: windows whose target falls in the last 60 steps are test data.
  3. Fits Ridge (linear regression with a small L2 penalty) and reports the one-step-ahead error.
  4. Forecasts all 60 test steps recursively from the end of the training data, feeding each prediction back as input, and compares with a "repeat the last cycle" forecast.
Python An autoregressive forecaster, one step and 60 steps ahead

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The results are strong. There are 216 training windows and 60 test windows. The one-step-ahead MAE is 0.375, less than half of the 0.829 of the seasonal naive baseline and much lower than the 2.37 standard deviation of the series. The recursive 60-step forecast, which never sees any of the real test values, has an MAE of 0.481, while simply repeating the last cycle scores 1.406 because the trend makes it fall further behind with every cycle. The plot shows the forecast tracking the cycles and the upward drift. A noise level of 0.4 means that the best possible one-step MAE is around 0.32 (for Gaussian noise, the mean absolute value is about 0.8 of the standard deviation), so 0.375 is already close to the floor.

Recursive forecasting has a built-in weakness. Every prediction feeds the next one, so small errors can compound. Here the series is so regular that this barely happens, but on a noisy, irregular series the forecast usually drifts toward a flat average as the horizon grows. Alternatives are to train a model that outputs the next H values at once (a direct multi-step model, which in PyTorch just means nn.Linear(hidden, H)) or to retrain regularly with fresh data.

Window size and honest splitting

Two more experiments. How long should the window be? And how much does it matter that we split by time? The editor below trains the same Ridge model with window lengths from 2 to 36, then compares the chronological split with a random split in which 60 randomly chosen windows are used for testing.

Python Window length and the leakage of a random split

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The window length matters a lot, and for a reason you can read off the series itself. With 2 lags the MAE is 0.968, with 6 lags 0.839, with 12 lags 0.489 and with 24 lags 0.375; 36 lags (0.380) adds nothing. The seasonal cycle has a length of 24, so the model needs to look back at least that far to see a whole period. A window shorter than the longest pattern cannot represent it. In practice, choose the window from what you know about the data (daily and weekly cycles, for example) and check a few values like this, not by guessing.

The splitting experiment shows how leakage flatters a model. With the honest, chronological split the MAE is 0.375; with a random split of windows it looks better, 0.334, because every test window has neighbors in the training set whose inputs and targets overlap it heavily, so the model has effectively seen the answer. The effect is modest here because the series is so regular. With real data, where trends, seasons and events change over time, a random split can be wildly optimistic. The rule is simple: the test set must come from after the training set.

For more careful evaluation, use walk-forward validation: train on the first part, test on the next block, then extend the training set to include that block and test on the one after, and average the errors. It mimics how the model will actually be used.

Extend the project

  • Add a second input feature (for example, a calendar value such as the position in the cycle, t % 24) and see whether the error drops. In PyTorch the LSTM input size becomes 2.
  • Make a direct multi-step model that predicts the next 12 values at once and compare its 12-step error with the recursive version.
  • Raise the noise to 1.5 in make_series and compare how the recursive forecast degrades against the baselines.
  • Add a sudden level shift halfway through the test period and watch what each forecaster does. This is how real-world forecasts fail.
  • In PyTorch, replace the LSTM with a nn.GRU, or with a 1-D convolution over the window, and compare the training time and error.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. A series of length 100 is turned into windows of length 10 with the next value as the target. How many examples are there?

    The first window uses values 0 to 9 and predicts value 10. The last target is value 99. That is 100 - 10 = 90 examples.
  2. 2. Why must the test set for forecasting come from after the training set?

    In deployment the model only ever sees the past. A chronological split reproduces that situation; a random split lets neighboring points reveal the answer.
  3. 3. Why does the 24-lag model clearly beat the 6-lag model on a series with a 24-step cycle?

    The window must cover the longest pattern you want the model to use. 24 lags span a full cycle; 6 lags do not.
  4. 4. What is the main weakness of recursive multi-step forecasting?

    The model's own mistakes are fed back as inputs. Direct multi-step models, which predict several outputs at once, avoid that feedback.

Summary

  • Windowing turns a series into supervised examples: L past values in, the next value (or the next few) out; a series of length n gives n - L examples.
  • Always compute baselines (last value, seasonal naive) first, and judge models against them rather than against zero.
  • Split by time, normalize with training statistics only, and prefer walk-forward validation for a more reliable estimate.
  • Choose the window length to cover the longest pattern in the data.
  • An LSTM in PyTorch uses (batch, time, features) input with batch_first=True and predicts from the last hidden state; recursive forecasting feeds predictions back, which can accumulate errors.
  • A simple autoregressive linear model is a strong, fast baseline, and often hard to beat on regular series.

Continue with Practice Exercises with Solutions.