Skip to content
Tutorials Center
My learning

Topics

Lesson 48 of 57 ยท Applied Statistics

Time Series Basics

Learn time series fundamentals: trend, seasonality, moving averages, autocorrelation, stationarity, differencing and exponential smoothing forecasts, in Python.

A time series is a sequence of measurements taken over time: daily sales, hourly temperature, monthly unemployment. It breaks the most basic assumption of everything you have used so far, independence. Today's sales are related to yesterday's, so ordinary formulas for standard errors and p-values can be badly wrong. Time series methods use that dependence instead of ignoring it.

Two goals dominate: describing the patterns (trend, seasonal cycles) and forecasting future values with an honest statement of uncertainty.

The components

Most series can be thought of as a combination of:

  • Trend: the long-run direction (growth or decline).
  • Seasonality: a regular pattern with a fixed period (weekly, yearly).
  • Cycle: slower, irregular swings (business cycles) with no fixed length.
  • Noise (irregular): random variation left over.

In an additive model y = trend + seasonal + noise; in a multiplicative one y = trend ร— seasonal ร— noise, which fits when seasonal swings grow with the level (taking logs turns multiplicative into additive).

Python Build and decompose a series

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The estimated monthly effects should rise and fall roughly like the true sine wave (amplitude 15), and the leftover noise is of the same order as the simulated standard deviation of 4 (the centered average itself absorbs a little of it).

Smoothing: moving averages

A moving average replaces each value by the mean of a window around it. Wider windows smooth more but react more slowly to real changes. A centered window is for describing the past; a trailing window (only past values) is what you can compute in real time.

Autocorrelation

The autocorrelation function (ACF) at lag k is the correlation between the series and itself shifted by k steps:

r_k = ฮฃ (yโ‚œ โˆ’ ศณ)(yโ‚โ‚œโ‚Šโ‚–โ‚Ž โˆ’ ศณ) / ฮฃ (yโ‚œ โˆ’ ศณ)ยฒ

Interpretation:

  • Slowly decaying positive autocorrelations indicate a trend or persistence.
  • Spikes at lags 12, 24, โ€ฆ in monthly data indicate yearly seasonality.
  • For pure noise ("white noise"), about 95% of autocorrelations lie within ยฑ1.96/โˆšn.
Python Autocorrelation of different series

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

White noise shows autocorrelations inside the band; the random walk is near 1 at every small lag; the seasonal series swings between positive (lag 12) and negative (lag 6, half a cycle).

Stationarity and differencing

Many methods need a stationary series: constant mean, constant variance, and autocorrelation depending only on the lag. A trending series is not. The usual cure is differencing: replace yโ‚œ by yโ‚œ โˆ’ yโ‚œโ‚‹โ‚ (the change). One difference removes a linear trend; a seasonal difference yโ‚œ โˆ’ yโ‚œโ‚‹โ‚โ‚‚ removes yearly seasonality. A formal check is the augmented Dickey-Fuller test (null: the series has a unit root, i.e. is non-stationary).

Two unrelated series that both trend upward will show a huge correlation (a spurious correlation). Always difference or detrend before correlating time series.

Python Differencing removes a trend and its false correlation

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The two independent series look strongly correlated in levels, which is meaningless; their changes are uncorrelated, as they should be.

Forecasting with exponential smoothing

Simple exponential smoothing forecasts the next value as a weighted average of all past values, with weights that decay geometrically: ลทโ‚œโ‚Šโ‚ = ฮฑยทyโ‚œ + (1 โˆ’ ฮฑ)ยทลทโ‚œ. A large ฮฑ (near 1) follows recent changes; a small ฮฑ smooths heavily. Extensions add trend (Holt) and seasonality (Holt-Winters). More general models are ARIMA (autoregressive, integrated, moving average) built from exactly the autocorrelation ideas above.

Evaluate a forecast on data it has not seen: keep the last part of the series as a test set (never shuffle randomly, because order matters) and measure errors such as MAE (mean absolute error) or RMSE. Compare against a naive forecast (tomorrow equals today, or the same day last week); a model that cannot beat it is not worth using.

Python Exponential smoothing vs a naive forecast

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Because the measurements are noisy around a slowly moving level, moderate smoothing beats the naive forecast, which copies the noise into its prediction.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why can't you use the usual independent-data formulas for a time series?

    Autocorrelation violates the independence assumption, so standard errors and p-values from ordinary formulas are often too optimistic.
  2. 2. A monthly series has an ACF spike at lag 12. What does that suggest?

    Values one year (12 months) apart are strongly related, which is the signature of yearly seasonality.
  3. 3. What does first differencing do?

    yt - y(t-1) is the change; for a linear trend the changes fluctuate around a constant, making the series closer to stationary.
  4. 4. How should a forecasting model be validated?

    Time order must be respected: fit on the early part, test on the later part, and compare with simple baselines.

Summary

  • A time series has trend, seasonality, cycles and noise; observations are dependent.
  • Moving averages smooth; the ACF reveals persistence and seasonality.
  • Stationarity (constant mean and variance) is usually reached by differencing; beware spurious correlation between trending series.
  • Exponential smoothing, Holt-Winters and ARIMA are standard forecasting families.
  • Validate forecasts on later data and beat a naive baseline.

Next: Multiple Testing and Effect Sizes