Skip to content
Tutorials Center
My learning

Topics

Lesson 26 of 40 ยท ARIMA and SARIMA

ARIMA Residual Diagnostics

Check whether an ARIMA model is adequate: residual ACF, the Ljung-Box test, normality checks, heteroskedasticity and what to change when the residuals still contain structure.

Fitting an ARIMA model always produces numbers, even when the model is wrong. Residual diagnostics tell you whether the model has captured everything that is predictable. The idea is simple: if the model has extracted all the structure, what is left over (the residuals, the one-step forecast errors) should look like white noise: no autocorrelation, constant variance, and a mean of zero. If the residuals still show a pattern, that pattern is information the model ignored, and you can improve the forecast by modelling it.

This lesson builds on ARIMA model fitting and choosing orders. It is the last step of the Box-Jenkins loop: identify, estimate, check, forecast.

Diagnostics check a model's adequacy, not its accuracy. A model can pass every residual test and still forecast poorly out of sample. Use forecast accuracy metrics for that.

flowchart LR
    A["Identify orders<br/>(ACF, PACF, AIC)"] --> B["Estimate the model"]
    B --> C["Check residuals"]
    C -->|"white noise"| D["Forecast"]
    C -->|"structure left"| A

The Box-Jenkins loop: diagnostics decide whether you forecast or go back and change the orders.

What to check

  1. Mean near zero. A biased model has residuals that sit above or below zero.
  2. No autocorrelation. The residual ACF should stay inside the confidence band at every lag, and the Ljung-Box test should not reject.
  3. Constant variance. Residuals should have a similar spread across time. Growing spread suggests a log transform or a volatility model.
  4. Roughly normal. Matters for prediction intervals, less for point forecasts.

The residual ACF and the Ljung-Box test

The Ljung-Box test combines the first h residual autocorrelations into one statistic. The null hypothesis is "the residuals are independent". A small p-value rejects it and says structure remains. Here we generate an AR(2) series and fit two models: one too simple, one right.

Python Under-fitted versus adequate model

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The AR(1) model leaves autocorrelation behind (a small p-value) while the AR(2) model passes. Note model_df: the test should subtract the number of fitted ARMA parameters from its degrees of freedom, so pass p + q.

Look at the residual ACF

A number is good, a list of the lags is better, because it shows which lag is the problem. A spike at lag 7 on daily data points to missing weekly seasonality; a spike at lag 1 or 2 points to a missing AR or MA term.

Python Residual autocorrelations by lag

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The built-in diagnostics

Every fitted statsmodels result has a summary() with the Ljung-Box (L1), Jarque-Bera (normality) and heteroskedasticity tests, and plot_diagnostics() draws four panels (standardized residuals, histogram with a normal curve, Q-Q plot and correlogram). You can read the same tests as numbers:

Python Normality and heteroskedasticity tests

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

What to change when a check fails

Symptom Likely cause Fix
Spikes at lags 1 to 3 in the residual ACF Orders too small Raise p or q, compare AIC
Spike at the seasonal lag (7, 12, 24) Missing seasonality Use SARIMA or Fourier terms
Variance grows with the level Multiplicative effects Log or Box-Cox transform before fitting
Variance comes in bursts Volatility clustering Model volatility separately (GARCH family, outside ARIMA)
A few huge residuals Outliers or events Add dummy regressors (ARIMAX)
Non-zero residual mean Missing drift or constant Add a trend/constant term

Do not over-fit the diagnostics

With many lags tested, one will occasionally cross the band by chance (about 5%). A lone marginal spike at lag 13 is not a reason to add a parameter. Prefer a parsimonious model that passes the overall Ljung-Box test, and judge candidates by out-of-sample error as well.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What should the residuals of an adequate ARIMA model look like?

    Residuals that still contain structure mean the model left predictable information unused. White noise is the goal.
  2. 2. The Ljung-Box test on the residuals returns p = 0.002. What does it suggest?

    The null hypothesis is independence. A small p-value rejects it, which means structure remains.
  3. 3. A residual ACF shows a large spike at lag 7 on daily data. Which change is most sensible?

    A spike at the seasonal lag signals unmodelled seasonality.
  4. 4. Does passing all residual tests prove a model will forecast accurately?

    A model can be adequate on the training data and still have large forecast errors on new data.

Summary

  • Residuals of a good model look like white noise: zero mean, no autocorrelation, constant variance.
  • Use the residual ACF to see which lag fails and the Ljung-Box test (with model_df) for one overall verdict.
  • Jarque-Bera and heteroskedasticity tests check normality and constant variance, which matter for intervals.
  • Each failure points to a fix: orders, seasonality, a transform, regressors or a volatility model.
  • Diagnostics do not replace out-of-sample evaluation.

Next: Multivariate Series: VAR and Granger Causality