AI
Why Hundreds of Tools Hurt Agent Performance
Prime Video’s shared MCP server accumulated hundreds of tools. Engineers saw agents struggling to pick the right three or four tools for a task. Context windows bloated. Hickman reported performance degraded and hallucinations increased. The paradox of choice from streaming apps had recreated itself for AI agents.
MCP Notifications as a Discovery Mechanism
The team used two MCP spec features. First: a tools list change notification that tells an agent its tool list has been refreshed. Second: the MCP session ID header introduced with streamable HTTP. After initialization, the server exposes only one tool: find tools. Tools are mapped to problem categories. The find tools description tells the agent how to request tools by category. The server tracks which problem space an agent is operating in via the session ID.
Demo: A Running Agent with Dynamic Tools
Hickman built a marathon training MCP server. With only find tools available, he asked for last year’s London Marathon finish time. The agent called find tools for “results” tools, received a notification, then fetched get_marathon_times. After getting the average time (4h 28m), he asked for a training plan. The agent called find tools again for “training” tools, which removed the results tools and loaded a single create_training_plan tool. The tool set changed mid-session without restarting the agent.
Benefits and Trade-offs
Only tools relevant to the agent’s current problem space occupy the context window. Adding a new tool does not penalize unrelated agent sessions. Tools can be added and removed mid-session. The approach works for both remote HTTP servers and local stdio implementations. Trade-offs include additional latency from re-fetching tools and the need for governance to avoid overlapping tool definitions.
Q&A
Does this work with off-the-shelf agents like Claude or Goose? Any MCP-compliant client can implement the same pattern; Amazon uses internal tooling but the spec allows it for any client. ▶ Watch (20:11)
When are tools removed from the agent’s catalog? Tools are removed when the client makes a new tools list request triggered by the notification; the server then returns only tools for the current category. ▶ Watch (21:51)
How does this compare to progressive disclosure with skills? The approach is tool-specific; there may be overlap, but MCP’s distribution and security benefits apply here. ▶ Watch (22:53)
How do you handle non-compliant clients? Amazon controls its internal clients, so compliance has not been an issue; external servers may face more challenges. ▶ Watch (23:51)
Could resources instead of tools be used for category discovery? Yes, resources could work, but using tools felt more natural for this case. ▶ Watch (24:51)
Notable Quotes
the same tools were being implemented over and over across teams. Billy Hickman · ▶ Watch (4:35)
We saw a performance degrade, hallucinations increase. Billy Hickman · ▶ Watch (5:02)
We’ve been able to dynamically mid-session switch between toolings available to the agent based on the problem category. Billy Hickman · ▶ Watch (14:02)
Even if you had really good descriptive names and other things, you still get to a threshold when you have too many and you still have to perform drops off and it’s much better when you just have three tools rather than 100. Lilia Abaibourova · ▶ Watch (18:53)
Key Takeaways
- MCP notifications and session IDs enable dynamic tool discovery without client changes.
- Exposing a single “find tools” tool keeps the initial context small.
- Tools can be swapped mid-session, reducing context blow and improving accuracy.
About the Speaker(s)
Billy Hickman is a Senior Software Engineer at Amazon Prime Video with over 10 years of experience building highly available, scalable distributed systems.
Lilia Abaibourova is a Principal Product Manager at Amazon Prime Video leading AI enablement for engineers. She has 15 years of experience building developer platforms and AI-first tools at Amazon, Peloton, HBO, and Microsoft.