AI
Why Agentic Traffic Breaks the Old Proxy Model
HTTP proxies were tuned for milliseconds and kilobytes. Agentic workloads blow past both constraints. Inference calls can run long, bodies are large, and the model name itself is buried inside the JSON payload, sometimes at the very end of the message. Proxies historically avoid reading request bodies because it costs memory and CPU. With MCP and the OpenAI responses API replacing chat completions, the interesting routing decisions now live exactly where proxies least want to look.
Two Identities, One Agent: The Authorization Problem
An agent carries two identities simultaneously: the user who delegated authority and the agent workload itself. Existing RBAC policies were not designed for that split. Envoy already enforces SPIFFE mTLS for workload identity inside Kubernetes and the same RBAC extension that has existed for years has been adapted for MCP requests. External processing filters handle body inspection and per-request authorization decisions. Token exchange work from Anthropic and Okta on top of OpenID Connect is still evolving, and Envoy’s enforcement layer will need to keep pace as those standards settle.
Dynamic Modules: Extending Envoy Without a Fork
Until recently, adding custom logic to Envoy meant running a sidecar process or forking and rebuilding the binary. Dynamic modules change that. A filter compiles into the same workload process with no separate hop. Envoy now accepts modules written in Go and Rust, not only C++. Avlasov said that requiring C++ mastery had historically forced contributors to walk away. The built-on-Envoy CLI handles toolchain setup so developers can focus on filter logic rather than Bazel configuration, which both speakers described as considerably worse than a dentist appointment.
Demo: OpenFGA Fine-Grained Access Control as a Go Dynamic Module
Hughberg wrote a Go dynamic module that integrates OpenFGA (fine-grained authorization) with Envoy AI Gateway. The filter extracts the model name and tool name from the MCP request body, then calls the OpenFGA server to decide whether that agent may use that tool. A code-assistant agent configured without create-issue permission gets blocked at the proxy before it can post proprietary Google data to a public GitHub issue, which was the exact failure Hughberg observed during prototyping when her agent consistently picked the wrong MCP server. The module was built using the built-on-Envoy CLI and Claude Code.
MCP Session State and the Road to Native Proxy Handling
Stateful MCP sessions require transferring state across a fleet of proxies when connections shift. Envoy now has a native solution for this, with a backup path if the primary approach does not fit a deployment. Envoy AI Gateway originally handled MCP sessions through external processing, which allowed fast iteration without waiting for Envoy release cadences. Once the native implementation is proven in production, the gateway will migrate to it. Avlasov called the external processing approach a valid velocity strategy for anyone building on top of Envoy who needs to ship before upstream support lands.
What Is Still Missing and How to Help
The Envoy maintainer group numbers between 40 and 50 people across multiple companies. Remaining gaps include full A2A transport coverage (agent card support is present but not all transports), OpenTelemetry semantic convention export for AI traffic, and a standard integration path for guardrail services, which today is largely bespoke per deployment. Avlasov and Hughberg asked users to state their use cases in GitHub issues and Slack. Google runs all agentic networking on Envoy internally. Tetrate runs inference and tool calls through a SaaS-hosted Envoy AI Gateway in production.
Notable Quotes
observability is the number one request that we hear from our our users. They want to know what agents do. Like everybody is paranoid at this point Yan Avlasov · ▶ 09:58
I would say that 95% or even the most important use cases that we saw are fully covered at this point. Yan Avlasov · ▶ 13:08
before we had to ask developers to know C++ and I always felt sad when a you know then somebody who wanted to contribute to envoy had to walk away because they didn’t have the mastery of C++ Yan Avlasov · ▶ 15:29
all of our infrastructure including agentic uh support for agentic networking is all envoy based within Google Yan Avlasov · ▶ 28:20
It’s uh it’s almost as good as me. Yan Avlasov · ▶ 31:31
Key Takeaways
- Agents carry two simultaneous identities; existing RBAC policies require adaptation to handle both.
- Dynamic modules let Go and Rust developers extend Envoy in-process without forking the binary.
- Tetrate and Google both run agentic traffic through Envoy in production today.
About the Speaker(s)
Yan Avlasov is a senior maintainer of Envoy proxy with eight years on the project. He works at Google helping teams deploy Envoy-based products and ensuring they meet Google’s standards for safety and security.
Erica Hughberg is a maintainer of Envoy AI Gateway at Tetrate, focused on features that let organizations route and control AI traffic through Envoy proxy and Envoy Gateway.