Scale your similarity search from one comparison to a real multi-document vector store, and see why the results make semantic sense even when the wording doesn't match.
1Why You Cache Embeddings Instead of Re-Computing Them
Generating an embedding costs a real API call every time. If your chatbot re-embedded every document in your knowledge base on every single user question, costs and latency would explode for no reason — the documents haven't changed. Production systems embed each document once at ingestion time, store the vector, and only embed the live query on each request.
2Top-K and Semantic Neighbors
Retrieval almost never returns just one result — you typically ask for the top few (top_k) matches, because the single best match might not contain everything needed to answer fully, and adjacent, related chunks often add useful context. That's exactly what happened here: a PTO question also pulled in the Parental Leave chunk, because both are 'leave policy' concepts sitting close together in embedding space.
3Step-by-Step Breakdown
Module 2: Vector Store & Retrieval. You now have real embeddings for three chunks. A real handbook has more sections than that — so this module scales up to five chunks and builds the actual search function your chatbot will call at query time: given a question, which chunks are actually relevant?
Search a Real Vector Store. These 5 vectors were generated once by calling the real embeddings API — exactly like you did last lesson — then cached here, because re-embedding unchanged documents on every search would be wasteful. Finish similarity_search(): sort scored so the best match comes first, then keep only the top_k ids.
Why does the vacation-days query also retrieve the Parental Leave chunk, even though the question never mentions parental leave?
- →Both are about time-off policies, so their embeddings sit close together in vector space — semantic similarity, not keyword overlap, drives the match.
- →Because 'parental-leave' comes right after 'pto-policy' in the dictionary's insertion order.
Raw Chunks Aren't a Prompt Yet. similarity_search() gives you back a list of chunk ids — not something you can paste into a prompt yet. Next lesson: formatting retrieved chunks into clean, labeled context the LLM can actually cite.
Level Up 🚀
Advanced cheat sheets, SEO tricks, and interview prep for this topic.
Browser Support
Fully supported.
Fully supported.
Fully supported.
Fully supported.
Accessibility (A11y)
1Expose Retrieved Source Count to Assistive Tech
When a RAG UI shows 'answered using 2 sources', make sure that count is in real text content, not just a visual badge, so screen reader users get the same transparency about grounding.
<span>Answered using 2 retrieved sources</span>SEO Implications
- 1
Target 'vector similarity search' and 'top-k retrieval' as distinct searches
Developers hit these as separate, specific implementation problems once past the basic embeddings tutorial stage.
Best Practices
Cache Document Embeddings, Only Embed the Live Query
Re-embedding unchanged documents on every request wastes API cost and adds latency for zero benefit — embed once at ingestion, store the vector, and reuse it indefinitely until the source document changes.
Frequent Bugs
Forgetting to re-sort after adding new documents to an in-memory vector store, so top_k results are stale.
Always compute similarity scores fresh against the current query — never cache search *results*, only the document embeddings themselves.
Real-World Examples
HR Knowledge Base
An HR chatbot's vector store returns both the PTO policy and the parental leave policy for a vacation question, giving the LLM enough related context to also proactively mention parental leave exists — a genuinely useful side effect of semantic (not keyword) search.
results = similarity_search(query_vector, top_k=2)