When More Tools Hurt Agent Performance

▶ Watch (2:17)

GitHub’s local MCP server launched in April 2024 with over 100 tools covering repos, issues, PRs, actions, and projects. Agents got worse, not better. Context windows filled quickly. LangChain research published in February 2024 showed the same pattern: more tools make agents confused and forgetful. Users could configure subsets via tool sets or dynamic selection. Almost nobody did. Everyone used the default settings. The team needed a different approach.

Context and Token Optimization at Scale

▶ Watch (5:11)

The team analyzed usage patterns on the remote server. They cut initial context load by 49% by focusing tools on common cases. Default tools dropped to 40. CRUD tools were grouped. Output tokens shrank. List pull requests lost 75% of its output tokens after tailoring exactly what came back. Tool failure rates fell below 5%. The server encodes agent intent into the tool surface. It makes multiple API calls server-side to reduce round trips and save context.

Security Trade-offs in Agent Authentication

▶ Watch (8:14)

Many users authenticate MCP with plain-text access tokens. Tokens are long-lived, over-privileged, and stored where agents can read them. GitHub’s remote server supports OAuth 2.1 with PKCE. The team helped add PKCE support to GitHub’s authorization server. Dynamic client registration was considered and rejected. It creates unbounded app database growth and unreliable app identity. The server filters tools by token scopes automatically. Step-up OAuth lets users upscope interactively without failing the tool call.

Stateless Server Design for Reliability

▶ Watch (13:48)

The server is completely stateless. A new server instance is created per request inside the SDK. Tools are added at startup based on user configuration and policy. Redis handles session storage. Sessions only track client identity. No session affinity is needed. The system handles 7 million tool calls per week. An Insiders mode exposes experimental features like MCP apps. Users can review AI-generated issues before posting.

Q&A

What is your take on piping tool calls? Sam suggested wrapping read-only tools in a CLI with proper help and letting agents use them, noting it is surprisingly effective. ▶ Watch (19:44)

Notable Quotes

More tools don’t make better agents, you know, they get confused and forgetful. Sam Morrow · ▶ Watch (2:33)

Everyone used the default settings. Sam Morrow · ▶ Watch (4:01)

we serve around 7 million tool calls a week. Sam Morrow · ▶ Watch (14:48)

Key Takeaways

  • More tools broke agent performance; GitHub cut context by 49% and tools to 40.
  • OAuth 2.1 with PKCE replaced plain-text tokens for secure authentication.
  • A stateless architecture with 7 million weekly calls proved MCP can scale.