Model Deployment for RecSys in AI & Artificial Intelligence

When a user opens an app, you can't score every one of your 10 million items in real-time. Instead, we use a Two-Stage pipeline. The first stage is Retrieval (or Candidate Generation), which uses simple, fast logic to find the top ~100 items most likely to interest the user. The second stage is Ranking, where a more complex and 'heavy' model (like a Deep Neural Network) scores only those 100 candidates to produce the final top-10 list shown to the user.

To make the Retrieval stage fast enough, we convert items and users into Embeddings (vectors) and use Approximate Nearest Neighbors (ANN). Algorithms like HNSW (Hierarchical Navigable Small World) allow us to search through millions of vectors in milliseconds by creating a navigable graph of similarities. This 'approximation' trades a tiny bit of accuracy for a massive gain in speed, which is the fundamental trade-off of production-grade Recommender Systems.

Building a recommendation engine is one thing; serving it to millions of users in real-time is another. In this lesson, we'll master the architecture for deploying RecSys at scale.

RecSys deployment usually follows a 'Two-Tower' or 'Multi-Stage' architecture: Retrieval (finding 100 candidates) and Ranking (scoring those 100 with a complex model).

For the Retrieval stage, we use 'Approximate Nearest Neighbors' (ANN) libraries like Faiss or Annoy. These allow us to find similar embeddings in logarithmic time.

Checkpoint: Why don't we use a complex Neural Network to score ALL millions of items for every user request?

• →It takes too much memory
• →It's too computationally expensive and would take seconds to respond, which is unacceptable for real-time apps

We also need to handle 'Near Real-Time' (NRT) updates. If a user clicks a video, we must update their profile immediately so the next recommendation reflects that interest.

Deployment also requires monitoring for 'Filter Bubbles'. If the model only recommends what the user already likes, they get bored. We inject 'Exploration' to keep the feed fresh.

Checkpoint: What is 'Exploration' in the context of a deployed recommendation engine?

• →Searching for bugs
• →Deliberately showing users new or different items to prevent them from getting stuck in a 'Filter Bubble' of the same content

By mastering deployment, you bridge the gap between a research model and a production system that provides value to users every second.

Pro-tip: Use 'Feature Stores' to serve consistent user and item features to both the training pipeline and the real-time inference engine.

Checkpoint: True or False: Approximate Nearest Neighbors (ANN) provides the exact same result as a brute-force search, but faster.

• →True
• →False

Recommendation infrastructure deployed! Your models are now scale-ready.

Next, we'll build our final Recommender Systems Capstone: a full Movie Recommendation Engine from scratch.

Check a Real Latency Budget. Finish checking whether an inference call stays within its real-time serving budget.