AI
Why the Service API Has Reached Its Limit
Service is one of the oldest Kubernetes APIs. It was designed 10 years ago and used by nearly every Kubernetes workload. Kubernetes maintainers will no longer accept contributions that just add more on top of it. The API does too much at once, and it is still constrained to L4. New protocols, AI workloads requiring GPU spillover across regions, and heterogeneous endpoint counts (a dozen pods in one cluster, two in another) have all pushed past what Service was built to handle.
The Front-End and Back-End Split Inside a Single Service
Service bundles two distinct things. The front end is the human-readable FQDN, the .cluster.local hostname, and the cluster IP. The back end is a bucket of endpoint IPs, typically pods. When you use a Gateway API HTTPRoute with a backendRef pointing at a Service, you only consume the back-end part. The gateway acts as the front end. Separating these two facets cleanly is the foundation for every proposal Morris and Grosskurth present. Forcing both facets to travel together is what makes multi-cluster extension so awkward today.
Lessons From Four Years With the Multi-Cluster Services API
MCS API introduced ServiceImport and ServiceExport. The model is sound, but adoption stalls on two friction points. First, switching from .cluster.local to .clusterset.local requires updating code across hundreds of workloads in large organizations. Second, namespace sameness, the assumption that foo in namespace bar on cluster A equals foo in namespace bar on cluster B, is hard to retrofit onto organizations where many teams each manage their own clusters with their own naming conventions. MCS API also has no mechanism for communicating endpoint load or GPU capacity, which matters acutely now.
Cluster IP Gateway and Composite Backend: The Proposed Data Model
Grosskurth’s cluster IP gateway gives a front-end Gateway a DNS name under gw.cluster.local instead of svc.cluster.local, backed by a VIP from the service CIDR range. A TCPRoute connects it to an EndpointSelector, a new resource that extracts just the pod-selector back-end logic from Service. For multi-cluster, a composite backend sits between the route and the endpoint selector. In a single cluster it is a no-op. Across clusters, the implementation merges back ends from every cluster sharing the same name and namespace, enabling load balancing without rewriting consumer addresses.
Prototype Implementation: XDS Instead of Kubernetes Object Replication
A working prototype was pushed to GitHub the morning of the talk. It ships a CoreDNS plugin that watches Gateway status addresses and synthesizes gateway.cluster.local DNS records, a controller that allocates service VIPs and creates endpoint slices, and a kube-proxy-style agent that programs nftables. For multi-cluster, the controller serves XDS instead of writing endpoint slice objects into every cluster. A hierarchy of XDS relays propagates endpoint information across clusters without materializing full Kubernetes objects everywhere, which is difficult to scale.
Where the Work Goes Next
The egress gateway proposal, driven by Morgan, Keith, and Shane in the Gateway API AI working group, independently converged on wanting the same backend resource and endpoint selector. Keith is also driving a backend resource proposal directly in Gateway API. The AI working group holds design discussions but owns no code. Proposals move from there into Gateway API’s experimental channel for faster delivery to users, and eventually into sig-network for upstreaming components like EndpointSelector that serve broader use cases beyond multi-cluster.
Notable Quotes
service is one of the oldest APIs. It is very much a load-bearing API. It is used by almost every user of Kubernetes, and it was designed 10 years ago. Mike Morris · ▶ 01:21
needing to update code to switch to a .cluster.set.local uh address instead of your .cluster.local is just it’s too heavy a burden for adoption, especially in large organizations Mike Morris · ▶ 05:52
inference pool import is basically a short-term hack. Mike Morris · ▶ 11:41
just pushed onto GitHub this morning Alan Grosskurth · ▶ 19:12
that’s how we know these things are going to stick and land and then they’re not just going to be a one-off thing that six months from now when the AI needs change gets discarded. Mike Morris · ▶ 24:14
Key Takeaways
- The Kubernetes Service API is L4-only, 10 years old, and no longer extensible by maintainers.
- Separating Service’s front-end hostname from its back-end endpoint bucket enables flexible multi-cluster routing without changing consumer addresses.
- A composite backend resource merges endpoint slices across clusters without replicating full Kubernetes objects into every cluster.
- XDS relays propagate multi-cluster endpoint data at scale, bypassing the single-cluster focus of the Kubernetes API.
- AI GPU spillover requirements are funding multi-cluster networking work that the community has wanted for years.
About the Speaker(s)
Mike Morris is a product manager at Microsoft focused on upstream open source networking, including Istio service mesh and Gateway API for service mesh, where he serves as co-lead. His work centers on building sustainable communities and scalable distributed systems.
Alan Grosskurth is a software engineer at Google Cloud working on containers, networking, and service meshes. He led the prototype implementation work presented in this talk, including the cluster IP gateway and the XDS-based multi-cluster extension.