TS Metrics in AI & Artificial Intelligence

MAE (Mean Absolute Error) is the simplest measure of error; it tells you the average 'raw' distance from the truth. RMSE (Root Mean Square Error), however, squares the errors before averaging them. This makes RMSE much more sensitive to Outliers. If your business loses a massive amount of money on a single large forecasting mistake (e.g., running out of inventory), you should optimize for RMSE to force the model to avoid those 'big misses.'

MAPE (Mean Absolute Percentage Error) is the most common metric for communicating with non-technical stakeholders. Knowing that your error is '50 units' is meaningless without context; knowing that your error is '5%' is immediately understandable. However, MAPE has a weakness: it can't handle zero values in the actual data, as you can't divide by zero. In such cases, we often use sMAPE (Symmetric MAPE).

Beyond the magnitude of error, we must look for Bias. If your model consistently predicts values that are *higher* than reality, it has a Positive Bias. If it predicts *lower*, it has a Negative Bias. We measure this using the Mean Error (ME). A good model should have balanced errors that average out to zero over time. Consistent bias usually indicates that your model is missing a key explanatory feature.

Is your forecast good or bad? In time series, we don't use 'accuracy'. We measure the distance between our prediction and the actual value using specialized error metrics.

MAE (Mean Absolute Error) is the most intuitive. It's the average of the absolute differences. If your MAE is 5, your forecast is off by 5 units on average.

RMSE (Root Mean Square Error) is more sensitive to large errors. It squares the differences, making outliers very 'expensive' for the model.

Checkpoint: Which metric would you use if you want to heavily punish occasional large forecasting errors?

• →MAE (Mean Absolute Error)
• →RMSE (Root Mean Square Error)

MAPE (Mean Absolute Percentage Error) is great for business reporting. It expresses error as a percentage, like 'Our forecast is 95% accurate' (5% MAPE).

No single metric is perfect. By tracking all three, you can understand if your model is generally accurate or if it's struggling with extreme events.

Checkpoint: If a model has a very high RMSE but a low MAE, what does this tell you about the errors?

• →The model is perfect
• →The model is usually accurate but makes a few very large mistakes

Metrics mastered! You've learned to quantify failure. Ready to build your first real statistical model: The AR (Autoregressive) model?

Compute Real RMSE. Finish implementing Root Mean Squared Error, the most common forecast accuracy metric.