Why is a RAG chatbot giving inaccurate answers, according to Gaurav Sen's AI interview #7?

In the mock interview on Gaurav Sen's channel, the candidate Gaurav rules out offline indexing and query rewriting, and calls a poor embedding model unlikely. The likely cause is retrieval of chunks marked relevant that are not really relevant. The team takes the top 20 to 25 chunks per query, which fills the context window, so the LLM must search a haystack for the answer and becomes more likely to hallucinate.

• Gaurav suggests the team may have raised top-k because relevant chunks were not surfacing, which made the pollution worse.

Source: our own transcription of the episode. The full conversation is on Gaurav Sen (2026-09-28). Read the whole episode summary →

▸More from this episode