What does a cross-encoder re-ranker do in a RAG pipeline?

In Gaurav Sen's mock AI interview, Gaurav explains that a re-ranker is a cross-encoder that skips embeddings. It takes the user query's tokens and each chunk's tokens together and returns a similarity score per chunk. With 20 to 25 retrieved chunks you get 20 to 25 scores, keep the highest, drop the rest, and pass the survivors to the LLM, which should raise accuracy and reduce hallucination.

• Each chunk, for example around 200 words, is scored individually against the single user query.

Source: our own transcription of the episode. The full conversation is on Gaurav Sen (2026-09-28). Read the whole episode summary →

▸More from this episode