Skip to content
Tutorials Center
My learning

Topics

Lesson 32 of 40 ยท Machine Learning Forecasting

Neural Networks for Forecasting

Forecast with neural networks: windowing a series, a scikit-learn MLP on scaled windows, how LSTM and other deep models fit in, and honest guidance on when deep learning beats simpler models.

Neural networks learn a flexible nonlinear function from inputs to outputs, which makes them natural candidates for forecasting. The recipe is the same as for the other machine learning models in this chapter: cut the series into windows (the last L values as inputs, the next H values as targets) and train a regressor. What differs is the model family and the care it needs: neural networks are sensitive to scaling, need more data than trees or linear models, and have more settings to tune.

This lesson uses scikit-learn's MLPRegressor, which runs in your browser, and explains how deeper architectures extend the idea.

Windows and scaling

A multilayer perceptron (MLP) takes the L most recent values as a flat input vector. Because networks train by gradient descent, inputs and targets should be on a small, common scale. Fit the scaler on the training portion only.

Python An MLP on scaled windows

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The network compares well with the seasonal naive baseline on this clean periodic series. On a noisy real series the margin is often much smaller, which is why the baseline always belongs in the table.

How much does scaling matter?

Python The same network without scaling

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Without scaling the optimizer struggles: scikit-learn warns that training did not converge, and the error (about 7.5) is barely better than the plain baseline (about 8.8), even though the noise level is only 8 on a signal of amplitude 100. The scaled network in the first example gets within the noise. Scaling is not optional for neural networks.

Predicting a whole horizon

Like random forests, an MLP can output a vector. Give it a target of H columns and it forecasts the whole path in one pass (the MIMO strategy), which avoids recursive error build-up.

Python A multi-output MLP (24 hours ahead)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Beyond the MLP: sequence models

These architectures are used in practice but need frameworks that do not run in this site's browser runner, so the code below is read-only:

Model Idea Notes
RNN / LSTM / GRU Reads the window one step at a time and keeps a hidden state Good for long, irregular dependencies; slow to train
1-D CNN / TCN Convolutions along time Fast, parallel, a strong baseline
N-BEATS / N-HiTS Stacks of fully connected blocks with trend/seasonal heads Competitive with statistical models on large collections
Transformers (PatchTST, TFT) Attention across time steps, with covariates Need lots of data; strong on many related series
Python
# Read-only sketch (Keras): a small LSTM forecaster. Not runnable on this site.
# model = keras.Sequential([
#     keras.layers.Input(shape=(L, 1)),
#     keras.layers.LSTM(32),
#     keras.layers.Dense(H),
# ])
# model.compile(optimizer="adam", loss="mae")
# model.fit(X_train[..., None], Y_train, epochs=30, validation_split=0.1)

The data preparation is identical to the MLP above: windows of scaled values, with the extra trailing axis expected by recurrent layers.

When are neural networks worth it?

  • Many related series (thousands of products, meters or sensors): one global network learns shared patterns and often beats per-series statistical models.
  • Rich covariates (price, weather, promotions, text) and long histories.
  • Nonlinear dynamics that ARIMA and exponential smoothing cannot express.

For a single series with a few hundred points, a seasonal naive baseline, ETS or ARIMA is usually as accurate and far cheaper. Large forecasting competitions repeatedly show that simple methods are hard to beat on small data, and that deep or boosted global models win when data is plentiful. Decide with a proper time series cross-validation, not with intuition.

Neural networks are random: seed them, and for important forecasts average several runs (an ensemble). Averaging also reduces variance and usually improves accuracy.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why must inputs be scaled before training an MLP?

    Gradient-based training converges slowly or poorly when features are on large, different scales.
  2. 2. On which data should the scaler be fitted?

    Fitting on test data leaks future statistics into training.
  3. 3. What does giving an MLP a target with H columns do?

    A multi-column target makes the network output the entire future path at once.
  4. 4. For a single monthly series with 150 observations, which is the sensible first step?

    Small data favours simple models; deep models need much more data to pay off.

Summary

  • Neural forecasters learn from windows of past values; scale inputs and targets, fitting the scaler on training data.
  • An MLP can output a whole horizon at once; RNNs, TCNs, N-BEATS and transformers extend the idea (read-only here).
  • They shine with many related series, covariates and long histories, not with tiny single series.
  • Seed and ensemble, and always compare against the baselines with time-ordered validation.

Next: Forecast Accuracy Metrics