Answered from AI Interview Questions #3 - Prompt Caching with LLMs · Gaurav Sen · 2026-08-03
Gaurav Sen gives two options. If the answer can go out of date quickly, as when a strike blocks the road between Bangalore and Mysore, each cached response gets a timeout, for example one minute. If staleness is unlikely, the cache can instead use least-recently-used (LRU) eviction and drop old prompts as space is needed.
Source: our own transcription of the episode. The full conversation is on Gaurav Sen (2026-08-03). Read the whole episode summary →