Autoregressive (AR) Models in AI & Artificial Intelligence

An Autoregressive (AR) model predicts the current value of a series by taking a weighted sum of its own previous values. The formula is $Y(t) = eta_0 + eta_1 Y(t-1) + dots + eta_p Y(t-p) + epsilon$. This is essentially linear regression, where the 'features' are simply lagged versions of the target itself. This makes AR models exceptionally good for data that exhibits Momentum or Mean Reversion.

Statistical models like AR assume that the rules governing the data don't change over time. This is called Stationarity. A stationary series has a constant mean and variance. If your data has a trend (it's going up) or seasonality (it repeats), it is Non-Stationary. You must 'transform' it—usually by Differencing (subtracting yesterday's value from today's)—to make it stationary before the AR model can work correctly.

How many lags should you use? We use the Partial Autocorrelation Function (PACF) plot. Unlike a standard correlation plot, the PACF shows the correlation between $Y(t)$ and $Y(t-k)$ *after removing the influence of all intermediate lags*. If the PACF 'cuts off' after 3 lags, it suggests an AR(3) model is the best fit. This prevents you from adding redundant features that would overfit the model.

The simplest way to predict tomorrow is to look at today. The Autoregressive (AR) model does exactly that: it uses previous values of the variable as predictors.

The 'p' in AR(p) stands for the number of 'lags' the model considers. An AR(1) model only looks at yesterday; an AR(7) model looks at the whole past week.

Before running an AR model, your data must be 'Stationary'. This means its mean and variance don't change over time. If your data has a trend, you must remove it first.

Checkpoint: What does the 'p' parameter represent in an AR(p) model?

• →Probability
• →The number of previous time steps (lags) used as predictors

We use the PACF (Partial Autocorrelation Function) plot to find the best value for 'p'. It shows which lags have a direct effect on the current value.

AR models are the foundation of statistical forecasting. They are simple, fast, and surprisingly effective for data with strong autocorrelation.

Checkpoint: Why must data be 'Stationary' before fitting an AR model?

• →To make it faster
• →To ensure the statistical properties of the data don't change, allowing the model to make reliable future predictions

AR modeling mastered! You've learned to build a self-predicting system. Ready to add 'Moving Averages' to create the powerful ARIMA model?

Predict with a Real AR(1) Model. Finish implementing the AR(1) formula: today's value depends linearly on yesterday's.