Skip to content
Tutorials Center
My learning

Topics

Lesson 17 of 28 ยท Time Series

ARIMA Models

Fit AR, MA and ARIMA models with Statsmodels, choose orders from ACF/PACF and AIC, check residuals with Ljung-Box, and produce forecasts with confidence intervals.

ARIMA is the standard statistical model for forecasting a single series from its own past. The name spells out its three parts, ARIMA(p, d, q):

  • AR(p), autoregressive: today's value depends on the previous p values.
  • I(d), integrated: the series is differenced d times to make it stationary.
  • MA(q), moving average: today's value depends on the previous q forecast errors.
flowchart LR
    A["Plot, ACF, ADF test"] --> B["Choose d (differences)"]
    B --> C["Choose p and q"]
    C --> D["Fit ARIMA(p,d,q)"]
    D --> E["Residuals look like noise?"]
    E -->|"no"| C
    E -->|"yes"| F["Forecast"]

The Box-Jenkins loop: identify, fit, check, repeat, forecast.

Fit an AR model

We simulate an AR(2) series with known coefficients, then see whether ARIMA recovers them. The class is statsmodels.tsa.arima.model.ARIMA.

Python Fit and read an ARIMA

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

ar.L1 and ar.L2 estimate the two coefficients; sigma2 is the noise variance. The const is the series' mean.

Choose the orders

Two helpers work together. ACF/PACF plots suggest candidates (Time Series Basics). Then compare a small grid of models by AIC or BIC: lower is better, and the penalty prevents overfitting.

Python Grid search by AIC

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The top-ranked models should be close to the true (2, 0, 0), sometimes with an extra term at nearly the same AIC. When two models are nearly tied, prefer the simpler one.

Check the residuals

If the model captured the structure, the residuals should look like white noise: no autocorrelation left. The Ljung-Box test checks this; its null hypothesis is "no autocorrelation", so you want a large p-value here.

Python Ljung-Box on the residuals

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The correct model passes (large p); the empty model fails (tiny p) because the dependence is still in its residuals. res.plot_diagnostics() draws the standard four plots (residuals, histogram, Q-Q and correlogram).

Differencing and forecasting

For a trending series, set d=1. The model then works on changes, and forecasts are automatically turned back into levels. res.get_forecast(steps) returns predictions with uncertainty; conf_int() gives the interval, which widens the further ahead you look.

Python Forecast a trending series

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The interval at step 14 is much wider than at step 1: uncertainty accumulates. Always show the interval with a forecast, and evaluate on a held-out block (here the last 14 days).

trend="t" adds a linear trend (drift) to a differenced model. Without it, an ARIMA(p,1,q) forecast flattens out to a constant level.

What ARIMA does not do

Plain ARIMA ignores seasonality and external drivers. For yearly cycles or promotions use SARIMAX. Older code may use statsmodels.tsa.ARMA or arima_model.ARIMA; those were removed, so use statsmodels.tsa.arima.model.ARIMA as shown here.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. In ARIMA(p, d, q), what does d control?

    d is the order of differencing, applied to remove trend and make the series stationary.
  2. 2. Which result for the Ljung-Box test on residuals is good news?

    Its null hypothesis is that residuals are uncorrelated; failing to reject it means the model captured the dependence.
  3. 3. Why do forecast intervals widen with the horizon?

    Uncertainty compounds as predictions are built on earlier predictions.
  4. 4. Two models have AIC 812.4 and 812.9. What should you do?

    Differences below about 2 are not meaningful; parsimony wins.

Summary

  • ARIMA(p, d, q) combines autoregression, differencing and moving-average error terms.
  • Choose orders from ACF/PACF, then compare candidates by AIC or BIC.
  • Residuals should be white noise: check with Ljung-Box and plot_diagnostics().
  • get_forecast returns forecasts and widening confidence intervals; validate on a held-out final block.

Next: Exponential Smoothing and SARIMAX