AI
Seven Years of Architectural Change: From 6 Microservices to Istiod
Istio 0.1 launched in 2017 with a design goal from Google and IBM: decouple routing configuration from proxy implementation. The original architecture included components like Mixer and Pilot that no longer exist. By 1.0, the control plane had grown from 3 to 6 microservices with 37 CRDs. Running a microservices control plane for microservices proved painful. Users needed a service mesh to manage their service mesh. By 2020, all components consolidated into a single binary called istiod, enabling canary upgrades where multiple versions ran in parallel within the same cluster.
Project Health: Quarterly Releases and 100 Active Contributors
At peak, the Istio project sustained over 150 monthly unique contributors and roughly 50 new contributors per month. That number has settled, but the project still sees 100 active contributors per month and 30 new contributors making their first contribution each month. The release side is equally strong. Version 1.1 took 11 months to ship for a three-month project. Since 1.2, the team has maintained a quarterly cadence with a maximum delay of 2 weeks on any release. That consistency lets users stay current on security patches and gives features room to progress.
LLM-Aware Routing with Gateway API Inference Extension
When running LLM workloads, models can weigh hundreds of gigabytes or terabytes, so requests must reach nodes that have the model loaded. Cold-cache tokenization can add seconds of delay, making prompt cache warmth a routing constraint too. The inference extension project introduced an “inference pool,” essentially a load balancer that handles LLM-specific routing. When a request arrives at the ingress gateway, Envoy forwards the full request body to an endpoint picker service. That service knows which nodes have the model loaded and which have a warm cache. It returns an IP and port, and Envoy routes there.
Ztunnel Gains a Life Beyond Istio
Ztunnel is Istio’s purpose-built ambient mesh proxy, written in Rust. It runs per node, handles L4 routing and mTLS, and outperforms full Envoy on throughput and latency due to its smaller codebase. Two CNI providers have adopted it outside the standard Istio setup. Calico ships a Tigera operator that installs Istio and a forked Ztunnel patched for its CNI’s network policy model. Cilium went further: it built its own control plane using Ztunnel’s simplified XDS protocol, added SPIRE for certificates, and wired the control plane to Ztunnel over a Unix domain socket. Both projects are contributing their changes back upstream.
Scalable Multicluster Networking with True End-to-End mTLS
Sidecar multicluster gave every proxy full global knowledge of every pod in every cluster. Customers hit scalability walls around 8 clusters. The previous workaround, multi-network mode with east-west gateways, broke end-to-end mTLS by terminating and re-establishing at the gateway hop. Ambient multicluster solves both problems. No per-pod data crosses cluster boundaries. Each cluster advertises only its east-west gateway address, service identities, and an approximate instance count. Cross-cluster traffic wraps an inner pod-to-pod mTLS connection inside an outer tunnel mTLS to the gateway. This beta feature landed in Istio 1.29, with GA targeting the August release.
Agent Gateway: Istio’s Third Proxy for Agentic Workloads
MCP is 15 months old. Every agentic protocol on the market is still unstable. Istio’s own APIs need to stay boring and predictable, so the project is not landing MCP there. Instead, Istio is integrating Agent Gateway as its third proxy, after Envoy sidecars and Ztunnel. Agent Gateway terminates mTLS like a waypoint and exposes purpose-built APIs for LLM routing, agentic protocols, centralized API key management, and token-based rate limiting per application. The integration ships as alpha in the upcoming 1.30 release.
Q&A
Does the ambient multicluster design still require an east-west gateway? Yes. The only way to skip the gateway is a flat network, and flat networks eventually exhaust private IP address space. ▶ 25:29
Is the scalability problem about IP addresses or global endpoint knowledge? Both matter, but the core issue is client-side load balancing: clients must know every available endpoint across clusters, which does not scale. ▶ 26:18
Notable Quotes
this is not cube scale. Mitch Connors · ▶ 19:21
uh I don’t want to call them liars, but Mitch Connors · ▶ 22:07
the ISTTO APIs get to stay boring. Mitch Connors · ▶ 22:50
Key Takeaways
- Istio consolidated from 6 microservices to one istiod binary, enabling canary upgrades with no full-cluster restarts.
- Ztunnel’s simplified Rust codebase lets CNI providers like Calico and Cilium embed it outside the standard Istio stack.
- Ambient multicluster shares no per-pod state across clusters, achieving both end-to-end mTLS and multi-cluster scalability.
- Agent Gateway becomes Istio’s third proxy, handling agentic protocols while Istio’s own APIs stay stable.
- The inference extension routes LLM requests to pods with the model loaded and prompt cache warmed.
About the Speakers
Mitch Connors is a Principal Software Engineer at Microsoft and serves on the Istio Technical Oversight Committee. Over 19 years he has worked at Google, F5 Networks, Amazon, an Industrial IoT startup, and State Farm Insurance.
Daniel Grimm is a Senior Principal Software Engineer at Red Hat. He joined Red Hat’s Istio team in 2019 and has served as a release manager several times across the project’s history.