Autoregressive Models: Time Traveling through Data
In linear regression, we predict Y using independent features X. In Time Series forecasting, an Autoregressive (AR) model predicts Y using its own past historical values as the independent features.
Mathematical Foundation
The core assumption of an AR model is that the current value of the series, $Y_t$, can be explained as a linear combination of its previous $p$ values (lags), plus a constant $c$, and white noise $\epsilon_t$.
Here, $\phi$ (phi) represents the coefficients (weights) that the model will learn during training using methods like Ordinary Least Squares (OLS) or Maximum Likelihood Estimation (MLE).
Partial Autocorrelation (PACF)
How do we determine $p$ (how many lags to look back)? If we look at standard Autocorrelation (ACF), a lag 1 effect might artificially inflate the correlation at lag 2.
The PACF isolates the direct correlation between $Y_t$ and $Y_&123;t - k&125;$ by removing the indirect effects of the intermediate lags. We typically choose $p$ based on the last lag in the PACF plot that significantly protrudes outside the confidence interval.
The Stationarity Prerequisite
Classical AR models demand that your data is stationary. This means the statistical properties (mean, variance) remain constant over time.
- If data has a trend, the model's coefficients will be biased.
- To fix this, we apply differencing: $Y_t' = Y_t - Y_&123;t - 1&125;$.
- Use the Augmented Dickey-Fuller (ADF) test to verify stationarity programmatically.
❓ Model Architecture FAQ
Why use AR instead of standard Linear Regression?
Standard regression assumes observations are independent. Time series data violates this inherently because today's price is highly dependent on yesterday's price. AR models are mathematically designed to exploit this temporal dependence.
What does "White Noise" ($\epsilon_t$) mean in the equation?
It represents the random, unpredictable variations in the data that cannot be captured by past lags. A perfect AR model will result in residuals (errors) that resemble pure white noise, meaning all signal has been extracted.