Why MCP Gateways Differ from API Gateways

▶ Watch (3:52)

An MCP gateway is all about tokens and state. Traditional API gateways handle routes and microservices. MCP gateways must manage the tokens an LLM burns and the tools it sees. Context is king. Exposing 300 tools or descriptions blows the context window. State matters too. The MCP protocol is stateful. Sessions between client and server must stay alive. That complexity is the first hurdle any gateway builder hits.

Tool Naming and Authentication

▶ Watch (7:30)

Multiple MCP servers expose the same tool name, like “create_issue”. A gateway must resolve collisions. Strategies include prefixing (staging vs. prod) or overwriting both name and description. Overwriting is powerful because it lets you optimize tool usage without modifying the server. On authentication, a gateway can pass through tokens, downscope them via token exchange, or swap tokens between identity providers. Downscoping limits what the LLM can do.

Meta Tools and Context Bloat

▶ Watch (11:01)

A general agent sees hundreds of tools and wastes context. Meta tools solve this. The gateway semantically indexes all frontend tools. It exposes a “find_tool” and “call_tool”. The agent searches for the right tool instead of parsing a giant list. Semantic search combined with keyword matching works best. The agent makes a few extra calls, but the context stays lean. Codeding agents already use this pattern. A gateway can provide it centrally.

Workflow Composition and Demo

▶ Watch (12:24)

Agents are non-deterministic and slow. Instead of letting an agent figure out a multi-step process, compose the flow yourself. Osorio demonstrated VMCP, a gateway that turns YAML workflows into single virtual tools. For a container image, one tool call gets the manifest, config, and SBOM in parallel. It then runs OSV vulnerability matching and GitHub searches. Claude called that single tool and returned a vulnerability summary. The agent never saw the underlying DAG.

Notable Quotes

“context is king” Juan A. Osorio · ▶ Watch (4:05)

“the user is what matters here and it’s not people, it’s the LLM” Juan A. Osorio · ▶ Watch (4:42)

“tools are first-class citizen” Juan A. Osorio · ▶ Watch (6:56)

“If you find a collision, take preference in this one.” Juan A. Osorio · ▶ Watch (8:27)

Key Takeaways

  • MCP gateways prioritize token efficiency and stateful sessions over route-based routing.
  • Tool name collisions are resolved by prefixing or overwriting at the gateway layer.
  • Meta tools reduce context bloat by semantically indexing and dispatching tool calls.

About the Speaker(s)

Juan Antonio “Ozz” Osorio is a Mexican software engineer living in Finland. His background spans security for OpenStack, Kubernetes, and bare metal environments. Currently at Stacklok, he founded the ToolHive project and has been building MCP infrastructure, including supply chain…