Answered from AI Interview Questions #3 - Prompt Caching with LLMs · Gaurav Sen · 2026-08-03
Yes, in Gaurav Sen's answer. Checking a semantic cache needs only an embedding model, which is fast and cheap to run. A real LLM call runs the prompt through all of the model's attention blocks, and since most teams use a hosted provider rather than running the model themselves, that call usually costs more.
Source: our own transcription of the episode. The full conversation is on Gaurav Sen (2026-08-03). Read the whole episode summary →