Skip to content
Tutorials Center
My learning

Topics

Lesson 75 of 111 ยท Beyond the Basics

Stationarity Tests and Exponential Smoothing

Test a series for stationarity with ADF and KPSS, find its seasonal period, stabilize its variance and forecast with Holt-Winters exponential smoothing.

The lesson on time series with machine learning introduced lag features, walk-forward evaluation, decomposition and SARIMA. This lesson adds the tools you use before choosing a model and a simple family of forecasters that often beat much fancier ones: exponential smoothing. You will learn to test whether a series is stationary, to read the two standard tests correctly, to find the seasonal period without guessing, to calm a growing variance with a log transform, and to forecast with the Holt-Winters method.

The parts of a time series

Most series can be described as a mix of these components:

  • Trend: a long-term rise or fall.
  • Seasonality: a pattern that repeats at a fixed, known period: every 7 days, every 12 months.
  • Cyclical variation: rises and falls with no fixed period, such as business cycles lasting two to ten years. It is hard to model because it is not periodic.
  • Irregular (noise): what is left over: random fluctuations and one-off events.

In an additive series the components add up (trend + season + noise) and the seasonal swing has a constant size. In a multiplicative series they multiply, and the swing grows with the level (holiday sales rise by 30%, not by a fixed 500 units).

What stationary means, and why it matters

A series is (weakly) stationary if its mean, variance and autocorrelation do not change over time. Many classical models, including ARIMA, assume it, because patterns learned from the past can only be projected into the future if the statistical behavior stays the same.

Three kinds of series behave very differently:

  • White noise around a fixed mean: stationary.
  • Trend-stationary: a deterministic trend plus stationary noise. Remove the trend and it is stationary.
  • Random walk (a unit root process): each value is the previous one plus a random step. It wanders, has no fixed mean and is difference-stationary: the first difference is white noise.

Testing with ADF and KPSS

The augmented Dickey-Fuller (ADF) test has the null hypothesis "the series has a unit root (is not stationary)". A small p-value (below 0.05) rejects it, so you may treat the series as stationary. The KPSS test reverses the roles: its null hypothesis is "the series is stationary (around a mean or trend)", so a small p-value suggests it is not. Running both avoids being fooled by either.

ADF says KPSS says Conclusion
Stationary (small p) Stationary (large p) Stationary
Not stationary Not stationary Not stationary: difference it
Stationary Not stationary Probably stationary around a trend: remove the trend
Not stationary Stationary Evidence is weak: look at the plot, get more data
Python ADF and KPSS on four kinds of series

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

White noise and the seasonal series pass both tests (ADF p is zero, KPSS p is 0.10 or more): a repeating seasonal pattern around a constant mean is stationary in the statistical sense, even though it is predictable. The random walk and the trending series fail both (ADF p about 0.84, KPSS p of 0.01 or smaller), and differencing, or removing the fitted line in the trend case, makes them pass. The ADF test has low power when the series is short or nearly a unit root, so do not use it alone, and do not skip the plot.

Finding the seasonal period

SARIMA and Holt-Winters both need the seasonal period. You can often read it off the calendar (7 for daily data with weekly habits), but when you are unsure, the data can tell you in two ways: the autocorrelation function peaks at multiples of the period, and the Fourier transform (see feature extraction) peaks at the frequency 1 / period.

A second problem is a variance that grows with the level. A log transform turns a multiplicative pattern into an additive one with constant swings, which is what additive models expect (the Box-Cox transform generalizes this).

Python Detecting the period and stabilizing the variance

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The autocorrelation of the differenced series is strongest at a lag of 24, a multiple of the true period, and the Fourier transform reads the period as exactly 12 steps. In the multiplicative series the changes become about 40% larger from the first to the last third (12.7 to 17.7); after the log transform the spread is the same at both ends (0.116 and 0.115). If you model the log series, remember to apply np.exp to the forecasts.

Exponential smoothing

Exponential smoothing forecasts by averaging the past, with weights that decay exponentially: the last observation counts most, the one before a bit less, and so on. A smoothing parameter between 0 and 1 controls how fast the memory fades. Three methods, each adding one component:

  • Simple exponential smoothing (SES) keeps a level. Its forecast is flat, so it suits a series with no trend and no seasonality.
  • Holt's method adds a trend (a smoothed slope), so forecasts follow a line.
  • Holt-Winters adds a seasonal component, additive or multiplicative, with a seasonal period.

The following example holds out the last two years of a monthly series with a trend and yearly seasonality and compares four forecasters against two baselines. statsmodels estimates the smoothing parameters by itself.

Python Simple, Holt and Holt-Winters against baselines

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The simple method is no better than the naive forecast, because a flat line cannot follow either the trend or the yearly wave. Holt follows the trend and improves, but it still misses the season. Holt-Winters models both and cuts the error from about 13.5 (naive) to 1.5, far better than repeating last year's values (7.0), which ignores the trend. The estimated smoothing parameters are close to zero because the pattern in this synthetic series never changes; on real data they are larger, which means the model adapts faster to recent values.

Use an additive seasonal component when the swing is constant, a multiplicative one (seasonal="mul", positive data only) when it grows with the level, or fit Holt-Winters to the log series. Exponential smoothing gives a strong, cheap baseline: if a complicated model cannot beat it on walk-forward validation (cross-validation for special data), prefer the simple one.

Machine learning models for time series

For one series with clear structure, ARIMA/SARIMA and Holt-Winters are hard to beat. When you have many related series, external inputs (prices, weather, holidays) or very long histories, switch to supervised learning with lag features and gradient boosting, or to neural sequence models: recurrent networks (LSTM and GRU), sequence-to-sequence models, one-dimensional convolutional networks and transformers. Neural models need much more data and tuning; they are not available in the in-browser runner here, but the preparation is the same: stationarity checks, outlier treatment (outliers), variance stabilization and a walk-forward test set.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What is the null hypothesis of the ADF test?

    ADF assumes a unit root. A small p-value rejects that and suggests the series is stationary. KPSS has the opposite null hypothesis.
  2. 2. A series passes the ADF test but fails KPSS. What is the most likely situation?

    Mixed results often mean trend stationarity: detrend the series (or difference it) and test again.
  3. 3. Which smoothing method is right for a series with a trend and a yearly pattern?

    Simple smoothing models only a level and Holt only level and trend. Holt-Winters adds the seasonal component.
  4. 4. Why take the logarithm of a series whose seasonal swings grow with the level?

    Log of a product is a sum, so growing percentage swings become constant-size swings. Remember to exponentiate the forecasts.

Summary

  • A time series mixes trend, seasonality, cyclical variation and noise; additive and multiplicative forms differ in whether the swings scale with the level.
  • Stationary series have a stable mean, variance and autocorrelation. Test with ADF (null: not stationary) and KPSS (null: stationary) together, and always look at a plot.
  • Difference to remove a unit root, detrend to remove a deterministic trend, and take logs to stabilize a growing variance.
  • Read the seasonal period from the calendar, the autocorrelation peaks or the Fourier spectrum.
  • Simple, Holt and Holt-Winters exponential smoothing add level, trend and season; Holt-Winters is a strong, cheap baseline.

Next: how to put a model behind a web endpoint with Serving Models with Flask and Render.