Skip to content
Tutorials Center
My learning

Topics

Lesson 23 of 40 ยท ARIMA and SARIMA

Choosing ARIMA Orders: Box-Jenkins and AIC

Pick p, d and q with the Box-Jenkins method: unit-root tests for d, ACF and PACF patterns for p and q, a grid search by AIC, and the parsimony rules that avoid overfitting.

Fitting an ARIMA model takes one line. Choosing its three numbers well is the real craft. The classic recipe, Box-Jenkins, has three stages: identify the orders from plots and tests, estimate the parameters, and check the residuals, looping back if they fail. This lesson makes each stage concrete, and then shows how a grid search by an information criterion can speed up the identification.

Step 1: choose d with a unit-root test

Differencing is about stationarity. Two tests answer the question from opposite sides:

  • ADF (augmented Dickey-Fuller): null hypothesis = the series has a unit root (is non-stationary). A small p-value means stationary.
  • KPSS: null hypothesis = the series is stationary. A small p-value means non-stationary.

Using both gives a sturdier verdict. A simple loop is: test, difference once, test again, and stop when the series passes.

Python Find d with ADF and KPSS

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The level series fails (ADF p is large, KPSS p is small); after one difference both tests agree it is stationary. So d = 1.

Step 2: read p and q from ACF and PACF

Work on the differenced series and use the table you learned with the AR and MA lessons.

Pattern after differencing Suggested model
PACF cuts off at lag p, ACF tails off AR(p)
ACF cuts off at lag q, PACF tails off MA(q)
Both tail off ARMA(p, q): start with (1, 1)
Both are tiny at every lag White noise: p = q = 0
Python Read the patterns numerically

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Both columns decay gradually instead of cutting off: the sign of an ARMA(1, 1) process. In real data the picture is messier, which is why we back the eye with a systematic search.

Plot the ACF/PACF with statsmodels.graphics.tsaplots.plot_acf and plot_pacf when you work locally. The shaded band is the same approximate 95% noise range we compute by hand here.

Step 3: search a small grid by AIC

The Akaike information criterion (AIC) rewards fit but charges a penalty per parameter; BIC charges more and prefers smaller models. Fit every combination of small p and q for the chosen d and compare. Keep the grid small (0 to 3) to avoid overfitting and slow fits.

Python Grid search p and q by AIC

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The true order (1, 1) should be at or near the top. Several nearby models usually score within 2 AIC points of each other; when that happens they are practically tied, so pick the simplest (fewest parameters).

Step 4: check, then simplify

A winner by AIC is only a candidate. Confirm that

  1. the residuals pass a Ljung-Box test (no autocorrelation left);
  2. every coefficient is significant, or removing it barely changes AIC;
  3. the roots are safe: if an AR coefficient is near 1 or an MA coefficient near -1, you are probably over- or under-differencing.

A common pitfall is cancelling factors: an ARMA(2, 2) can fit as well as an ARMA(1, 1) because one AR root cancels one MA root, which makes the estimates unstable. When both the AR and MA orders go up together without improving AIC, go back down.

Automatic order selection

Tools such as pmdarima.auto_arima automate the whole search (including seasonal orders), but they are not available in every environment and you should know what they do: they run unit-root tests to pick d, then a stepwise search over p and q scored by AIC. The loop you wrote above is the same idea. A hand-rolled version is often enough and is easy to inspect:

Python
best = min(
    ((p, d, q) for p in range(4) for q in range(4)),
    key=lambda order: ARIMA(y, order=order).fit().aic,
)

Always inspect what an automatic search returns; it can pick a model that wins by 0.5 AIC and fails the residual test.

Rules of thumb for parsimony

  • Prefer d of 0, 1 or 2 (rarely 2). Use the unit-root tests, not a hunch.
  • Keep p and q at 3 or less for non-seasonal data unless the ACF clearly demands more.
  • Compare models only when they use the same differencing and the same data; AIC is not comparable across different d.
  • When in doubt, forecast on a held-out block and compare errors (see forecast accuracy metrics).

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. The ADF test on a series gives p = 0.62. What do you conclude?

    ADF's null hypothesis is a unit root. A large p-value fails to reject it, so the series should be treated as non-stationary and differenced.
  2. 2. On the differenced series, the PACF cuts off after lag 2 and the ACF decays slowly. Which model do you try first?

    A PACF cutoff at p with a tailing-off ACF is the AR(p) signature.
  3. 3. Why is BIC often preferred over AIC when you want a compact model?

    BIC's penalty grows with the sample size, so it favors smaller models than AIC does.
  4. 4. Two ARIMA candidates have AIC values 812.4 and 813.1, and the second has fewer parameters. Which do you choose?

    AIC differences below about 2 mean the models are practically tied, so parsimony decides.
  5. 5. Why should AIC values not be compared across models with different d?

    Differencing changes the series being modeled (and drops observations), so the likelihoods are not computed on the same data.

Summary

  • Choose d with unit-root tests (ADF and KPSS), differencing until the series passes.
  • Read candidate p and q from the ACF/PACF of the stationary series.
  • Confirm by a small grid search on AIC/BIC; prefer the simplest model within about 2 points of the best.
  • Always finish by checking residuals; beware cancelling AR/MA factors and over-differencing.

Next: SARIMA: Seasonal ARIMA