How does Gaurav Sen speed up vector database search for a RAG chatbot?

Gaurav Sen suggests tuning the approximate nearest neighbour index, such as HNSW or IVF, then running the search in parallel and pooling database connections. He adds a Redis cache with vector search for common queries, horizontal scaling, and sharding the vector database so each query is routed to the shard that matches its type.

• Gaurav Sen estimates a million documents at five to ten chunks each means about 10 million vectors searched per query.

Source: our own transcription of the episode. The full conversation is on Gaurav Sen (2026-10-05). Read the whole episode summary →

▸More from this episode