Skip to content
Tutorials Center
My learning

Topics

Lesson 20 of 40 ยท ARIMA and SARIMA

Autoregressive (AR) Models

Learn how an autoregressive model predicts a series from its own past, when it is stationary, how the ACF and PACF reveal its order, and how to fit and forecast in Python.

Many series have memory. If today's temperature is high, tomorrow's is probably high too; if a sensor drifted up a moment ago, it will likely still be up now. An autoregressive (AR) model captures that memory in the simplest possible way: it predicts the current value as a weighted sum of the previous values plus a fresh random shock. The "auto" means the series is regressed on itself.

AR models are the first of the two building blocks of ARIMA (the other is the moving average). Understanding them well makes every later model easier to read.

The AR(p) equation

An AR model of order p, written AR(p), says:

y[t] = c + phi1 * y[t-1] + phi2 * y[t-2] + ... + phip * y[t-p] + e[t]

  • c is a constant.
  • phi1 ... phip are the autoregressive coefficients: how strongly each lag pulls on today's value.
  • e[t] is white noise: independent shocks with mean 0 and constant variance.

The simplest case is AR(1): y[t] = phi * y[t-1] + e[t]. If phi is 0.8, each value keeps 80% of the previous one and adds a new surprise. If phi is 0, the series is pure noise. If phi is negative, the series flips sign from step to step.

When is an AR process stationary?

An AR(1) process is stationary only when |phi| < 1. At phi = 1 you get a random walk, which wanders without returning to a mean and has no fixed variance. For AR(p) the condition is that all roots of the characteristic polynomial lie outside the unit circle; in practice, statsmodels checks this for you. This is why stationarity matters before you fit: an AR model only describes a series that has a stable mean and variance.

flowchart LR
    A["Shock e[t-2]"] --> B["y[t-2]"]
    B -->|"phi"| C["y[t-1]"]
    C -->|"phi"| D["y[t]"]
    E["Shock e[t-1]"] --> C
    F["Shock e[t]"] --> D

In an AR(1) process an old shock fades geometrically: after k steps only phi^k of it remains.

Simulate and look at the memory

The best way to feel phi is to simulate. The code below generates AR(1) series with different coefficients and prints the lag-1 autocorrelation, which for an AR(1) process equals phi.

Python AR(1) with different coefficients

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

For phi = 0.9 the autocorrelations shrink slowly (about 0.9, 0.81, 0.73): the shock's echo lasts a long time. For phi = -0.7 they alternate in sign. This geometric decay is the fingerprint of an AR process in the ACF.

Identify the order: ACF versus PACF

How do you know whether to use AR(1) or AR(3) on real data? Two plots help.

  • The ACF (autocorrelation function) shows the correlation between y[t] and y[t-k], including all the indirect influence through the lags in between. For an AR process it decays gradually.
  • The PACF (partial autocorrelation function) shows the correlation at lag k after removing the effect of lags 1..k-1. For an AR(p) process it cuts off after lag p.

So the rule is: PACF cuts off at p and the ACF tails off means AR(p).

Python PACF cuts off at the true order

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Lags 1 and 2 are far outside the band; lags 3 onward fall inside it. The PACF has told us the order is 2.

Fit an AR model and forecast

statsmodels offers AutoReg for pure AR models, fitted by ordinary least squares. It works on a pandas Series with a date index, which also gives you dated forecasts.

Python Fit AutoReg and forecast

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The estimated coefficients land close to the true 0.6 and -0.3. Notice that the forecasts shrink toward zero (the series mean) as the horizon grows: an AR model forgets where it started, and a long forecast is simply the mean.

Set trend="n" only when the series is already centered. For real data, keep the default trend="c" so the model estimates a constant, or subtract the mean first.

Choosing p with an information criterion

The PACF gives a candidate; you can confirm it by fitting several orders and comparing AIC or BIC. Lower is better, and both penalize extra coefficients so that the model does not overfit. ar_select_order automates the search.

Python Pick the order automatically

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Common mistakes

  • Fitting a non-stationary series. A trending series gives a phi near 1 and meaningless long-run forecasts. Difference it first; the next lessons show how.
  • Reading the ACF alone. An AR(1) and an AR(5) both show a decaying ACF. The PACF is the plot that tells them apart.
  • Using too many lags. Every extra lag costs a parameter and data. Prefer the smallest order whose residuals look like white noise.
  • Trusting long forecasts. AR forecasts decay to the mean, so beyond a few multiples of the memory length they carry little information.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. In an AR(1) model with phi = 0.8, what share of an old shock remains after 3 steps?

    Each step multiplies the leftover effect by phi, so after k steps it is phi to the power k: 0.8 cubed is about 0.51.
  2. 2. Which pattern in the ACF and PACF points to an AR(2) process?

    A pure AR(p) process has a PACF that cuts off at p, while its ACF decays geometrically or in damped waves.
  3. 3. Why is an AR(1) process with phi = 1 a problem?

    At |phi| = 1 shocks never fade, the variance grows with time, and the AR stationarity condition fails.
  4. 4. What does a long-horizon forecast from a stationary AR model converge to?

    Because the memory fades, far-ahead predictions revert to the long-run mean of the process.

Summary

  • An AR(p) model predicts y[t] from its own previous p values plus white noise.
  • It is stationary when its coefficients keep the process from exploding; AR(1) needs |phi| < 1.
  • The PACF cuts off at lag p; the ACF tails off. Confirm the order with AIC or BIC.
  • AutoReg fits AR models; forecasts revert to the mean as the horizon grows.

Next: Moving Average (MA) Models