The Context Window Problem with Naive MCP

▶ Watch (03:01)

Cloudflare’s OpenAPI spec is 2.3 million tokens. Convert it to MCP tools naively and you’re at 1.1 million tokens, a number no current model context window survives. Carey hit this wall trying to expose the full Cloudflare API, now at 2,600 endpoints across 16 product areas. The obvious fix was splitting into 16 separate MCP servers. That cut context, but users had to pick the right server manually, and coverage was still incomplete: six tools for a product suite that had 30 endpoints.

Three Approaches to Progressive Tool Discovery

▶ Watch (05:25)

Three approaches exist for loading only the tools an agent needs. CLI access lets agents call wrangler --help, read the command tree, and drill down, but requires shell access on the user’s machine. Tool search, as Claude Code implements it, keyword-matches the user’s question and loads around eight tools; about 1,600 tokens fire for every one that gets used. Code mode skips the tool list entirely. Generate a typed TypeScript SDK from the OpenAPI spec, give the model a single “code” tool, and let it write against the types. One tool replaces hundreds.

Running Untrusted Code Safely in Isolated Sandboxes

▶ Watch (09:40)

Letting an LLM write code against secrets-holding infrastructure would have been a textbook vulnerability two years ago: file system reads, secret exfiltration, infinite loops, crypto miners. The fix is dynamic Workers: each a V8 isolate, process.env absent by default, network blocked unless you flip a boolean. The Cloudflare MCP live demo (13:28) connected to all 2,600 endpoints in one auth flow. The WorkerD, Deno, and Pydantic Monty sandbox comparison (15:39) ran the same generated-code pattern across three runtimes.

What the Agent-Native API Future Looks Like

▶ Watch (17:33)

Two shifts are coming. On the server side: APIs need rate limiting built for AI agents. A single agent can loop across multiple sandboxes and hammer an endpoint repeatedly.

On the client side: more clients will emerge (there are very few well-used ones now), and they will save generated code as reusable scripts. Cron jobs that auto-repair when web scraping breaks. Actions users replay without re-prompting. MCP itself will become a one-flag toggle in every TypeScript full-stack framework, exposing any API as an agent tool.

Notable Quotes

untrusted code is mega mega scary Matt Carey · ▶ 9:41

We just let the model write code Matt Carey · ▶ 8:54

APIs have to be ready to take a beating Matt Carey · ▶ 17:16

Key Takeaways

  • Exposing a full API as naive MCP tools can push agent context past 1.1 million tokens.
  • Dynamic Workers run LLM-generated code in V8 isolates with no secrets or network access by default.
  • MCP will become a one-flag toggle in TypeScript frameworks, making every API agent-accessible.