Technology
AI Interview #9 - Why is the Agent so slow?
Gaurav Sen · Gaurav Sen · Transcript & summary by WhipScribe
A compact walkthrough of the questions to ask and the first fixes to try when an LLM agent with several tools feels slow.
Gaurav Sen · Gaurav Sen · Transcript & summary by WhipScribe
A compact walkthrough of the questions to ask and the first fixes to try when an LLM agent with several tools feels slow.
Tap a chapter to open that moment in the transcript; the link to it is copied for you.
“So that it will solve two problems with one type of solution.” Tanishk · at 3:50 — open in the transcript →
“a single agent system with certain set of tools is more than enough.” Tanishk · at 7:05 — open in the transcript →
In this seven-minute mock interview on Gaurav Sen's channel, a candidate named Tanishk is handed a vague complaint: a company's performance-review agent takes far too long on every query. Before proposing anything, Tanishk pins down the setup. It is a single agent on a hosted LLM reached through API calls, with about five tools for employee data and git commits exposed over MCP, and no memory layer beyond its own context.
The fixes Tanishk offers fall into two groups. The first is about how tools run: tools called synchronously wait on one another, so independent calls should go async, and an MCP server that exposes ten tools should be filtered to the five a department actually needs, which also trims context bloat. The second is about whether a tool call is needed at all: slow-changing employee facts such as leave balances can be cached and placed in the system prompt, and two tools with heavily overlapping work can be merged into one.
When the interviewer raises batching calls across multiple agents, Tanishk pushes back. A multi-agent design would add short-term and long-term memory and other complexity, and a single agent with a fixed set of tools is enough for a performance-review job.
The candidate in this mock AI engineering interview, who plays the engineer asked to diagnose and speed up a company's slow performance-review agent.
In Gaurav Sen's mock interview, candidate Tanishk traces a slow performance-review agent mainly to how its tools are called. The agent uses about five tools, and when they run synchronously each one waits for the previous response before the next starts. Extra tools pulled in from an MCP server, and tools that fetch data which could have been cached, add further delay and bloat the model's context.
Tanishk argues in Gaurav Sen's AI interview series that independent tool calls should be asynchronous. With synchronous calls, a second tool waits on the first, so a query that needs all five tools pays for each response in sequence. Tools that do not depend on each other's output can run async, while tools that are dependent and share a lot of work can be merged into one.
Yes, in Tanishk's answer on Gaurav Sen's channel. A company MCP server might expose 10 tools while one department's agent needs only 5, so the agent should be given a filtered list before calling the server. That reduces latency and also saves context, which matters when the agent has no memory beyond its own context window.
Tanishk's fix in Gaurav Sen's mock interview is to avoid the call when the data rarely changes. Employee facts such as the number of leaves can be stored in the backend ahead of time and passed straight into the system prompt as an extra instruction. The same review applies to every tool: some do mundane work whose output could simply sit in the model's context.
No, according to Tanishk in Gaurav Sen's AI interview #9. Asked about batching calls from several agents to the same tool, Tanishk says a multi-agent setup brings extra complexity, including short-term and long-term memory to maintain. For a performance-review agent, a single agent with a fixed set of tools is enough.
Our own transcript of the public episode, with a timestamp about every 90 seconds. Tap a timestamp or the link button on a paragraph to copy a link that opens this page there.
0:00So Tanishk, we are using AI to help do performance reviews in our company. This AI is very very slow. We find it difficult to use it. Anytime we make a new query to it, it takes long long long time. Can you help us review performance? Yeah. Okay, so when you say AI or a bot, are you mentioning a rack-based chatbot or an agent-based chatbot or agent based chatbot? Which one is it? So it's not so much a chatbot and it is an agentic application which does yeah, it doesn't you can't talk to it but you can give prompts to it. Okay, it's an agent basically. So now there can be multiple things where it can be slowing down because you will need employee data yes that also we take commits also yeah some git commits right yeah some git
0:50commits etc like you might be having some documentation roughly if I imagine you will be having around five tools is this a fair number to assume yes yes so let's say you currently have five tools with an LLM working behind the scenes. So one thing more to clarify, are you using an API calling type of service or are you hosting an LLM? No, we are using API calls. You're using API callings. Do you also have memory architecture or something like that? Or is it a single agent system? It's a single agent system. So you're not maintaining any memory anywhere else? No, we have the whatever memory is there, whatever context is there in that agent only. Okay, so you are only doing context engineering sort of thing, right?
1:36So it's context engineering thing. So there is no extra memory layer attached to this? No, no, we don't have any preference to something. So when we are designing the tools, right, either I have to make it sync or either I have to make it async. This is, let's say, tool 1. This is tool 2, right? And you need to call both of these tools, right? But if they are in sync, tool 2 will be waiting to get a response from tool 1, right? So there is a wait time here but let's say for your use case in particular I don't think you require sync tool functionality is it something that you are using currently I think we are having sync we haven't looked at async so most likely we are using sync oh yeah then this might be one of the issues you know which will be causing you a lot of
2:23time because if you have let's say five tools and for some use case for one of the issues, you know, which will be causing you a lot of time because if you have, let's say five tools and for some use case, for one of these use case, you require all the five tool calling T3, T4 and T5, right? So, you know, if they are in sync, you will have to wait, you know, for every tool response to get and then next tool will be called, right? So this will cause you a lot of problems with latency. know for every tool response to get and then next tool will be called right so this will cause you a lot of problems with latency so this is problem number one this is P1 right that it they are in sync so the tools we have are actually
2:56MCP server tools it is github or employee data also we have exposed it using MCP now can these tools also please loop? Are they not asynchronous by default? No, so if you are using a MCP server you then have to check if they have a sync or async functionality. Sometimes what happens is a single MCP server might have 10 different tools but you actually require only 5 of them. So there might be a MCP server that you would have developed for your company which will have all the tools but for your department you will only require some five tools right. So you will have to do a filtering sort of thing before calling the MCP server that these are the only tools which I need. So that will also reduce some amount of
3:41latency and it also reduces context bloat because if you are not calling all the tools then you will be saving some amount of context because you are not managing any memory here. So that it will solve two problems with one type of solution. Okay but what if one of these tools is especially slow, let's say T3 is very slow. So let's take for example employee data right so now employee data is not changing every day right only some of the things might be changing some of the things can be cached here which are you know set patterns like the number of leaves that they have you can store it even before calling the tool and all of that you can store it in the backend already right and you can use that value directly rather than calling the tool and getting the employee data
4:28and all of that. You can store it directly, you can get directly give that in the context itself, in the system prompter, you can give it as an extra instruction, which is going to the agent. So that is one way that you can, you know, save the time to call this particular tool. You can also configure one more thing that do you actually require five tools that is also one thing to consider while building these kind of solutions because some of these tools might be you know just doing some mundane work or some sort of work which you can automatically get inside the context of the model right away rather than you know calling a tool getting response, making it sync and all of that. Okay. Two things that I've heard and I've only heard of this,
5:08I'm not sure if it actually happens. One is, can we compose these tools together so that they run together? If it is waiting on the result like you showed T1 is waiting for, or T2 is waiting for T1, then can it be that T1 and T2 can be merged together as a single tool? Okay, yeah, let's discuss this. Rather than merging, if let's say these two tools are doing some work which is, you know, very common. So, there is a high intersection. Let's say this is the task of T1 and this is the task of T2 and you have a lot of intersection, right? Then you can probably merge it into a single tool which will be doing both the tasks. This is one way or if you think that you know I need all the five tools and these are very
5:52independent of each other then you need to introduce a sync if they are not dependent on each other's output. Yeah but okay if they are dependent and they are back to back then you can compose the tools too. Yes you can compose if it has a lot of intersection. Let's say if T1 is fetching some employee data from one server and T2 is fetching some other data, but both of these data have a lot of intersection, a lot of things which are common, then you can compose it. Great. What about the second thing I heard of is batching. Is there a possibility of batching all the tool calls of a particular type, let's say T3 again or T4, like can it be that multiple agents are calling the same tool and instead
6:33of answering them one by one, we do this in software engineering, we batch all the calls together and get the response in batch. Is that something which is possible or not possible? For agents, when you're suggesting multiple agents calling the tools, that is a multi-agent setup. We are not, and in the multi-agent setup, you'll introduce some more complexities, like you will have to maintain a memory. You'll have to, if it's even a chatbot sort of thing, you'll have to maintain a short-term memory and a long-term memory. So you will introduce a lot of complexity in the thing. I think for this kind of thing, for a performance review agent, a single agent system with certain set of tools is more than enough.
7:09Amazing. Awesome. Brilliant. Wonderful. Is there a tool for increasing salary for after performance review? Let me know. There is a tool for decreasing. You know, how did your profile get shortlisted? I want to find out. The HR board working are under you, right? We will get back to you soon. Sure, but my resume is with you only. Okay, thank you.