RecSys Metrics in AI & Artificial Intelligence

In RecSys, we don't care about the 'Whole list'—users only look at the top few items. Precision@K tells us what percentage of the items in the top 'K' slots were actually relevant. Recall@K tells us how many of the available relevant items we successfully captured in that same window. There is always a trade-off: as you show more items (increasing K), Recall goes up, but Precision usually goes down because you're including lower-quality matches to fill the slots.

Normalized Discounted Cumulative Gain (NDCG) is the most important metric for production systems. Unlike Precision, which treats every slot as equal, NDCG is Rank-Sensitive. It uses a logarithmic 'Discount'—an item at position #1 is worth significantly more than an item at position #10. This encourages the algorithm to be extremely confident about its top-most choices, perfectly matching the human behavior of scanning lists from the top down.

A system with 100% Precision might actually be a bad product. If a user likes 'Star Wars', a 100% precise system might only recommend 'Star Wars 1-9'. This is accurate but Boring. Professional systems also track Diversity (are the items different from each other?) and Novelty (how 'Unexpected' or 'Unknown' is the recommendation?). The ultimate goal is Serendipity—finding something the user didn't know they wanted, but absolutely loves once they see it.

How do you know if your recommendations are actually good? Measuring success in RecSys requires more than just accuracy; it requires understanding ranking, relevance, and user satisfaction.

Precision@K measures how many of the top K items were actually relevant. Recall@K measures how many of the total relevant items we successfully found.

NDCG (Normalized Discounted Cumulative Gain) is the industry gold standard. It rewards the system for putting the *most* relevant items at the very top of the list.

Checkpoint: What does 'Precision@10' measure?

• →Total items in the database
• →The percentage of relevant items within the first 10 recommendations shown

We also measure 'Novelty' and 'Diversity'. If you only recommend what the user already knows, your NDCG might be high, but the user will eventually get bored.

By mastering these metrics, you move from 'Guessing' to 'Engineering', using data to prove the value of your recommendation algorithms.

Checkpoint: Why is NDCG often preferred over simple Precision?

• →It's easier to calculate
• →Because it cares about 'Ranking'—it gives more points for a relevant item at position #1 than at position #10

Evaluation metrics mastered! You've learned to measure. Ready to test in the real world with A/B Testing?

Compute Real Precision@K. Finish computing what fraction of the top-K recommendations were actually relevant.