AI
Multi-tenancy Requirements in Production Kubernetes
As Kubernetes workloads move to production, organizations share compute, storage, and networking across tenants while requiring strict isolation. Four requirements emerge: physical separation of compute nodes to prevent side-channel attacks, logical access boundaries through namespaces and network policies, per-tenant control planes with independent API servers, and a native Kubernetes experience where each tenant manages its own CRDs, cluster roles, and storage classes without awareness of the shared infrastructure underneath.
Three Existing Models and Their Gaps
Cluster-as-a-service gives each tenant a full cluster. Isolation is strong, but managing hundreds of clusters becomes a significant operational burden. Namespace-as-a-service, used by Capsule (a CNCF sandbox project), confines tenants to namespaces. Tenants still can’t use cluster-scoped resources like CRDs, and the noisy neighbor problem persists on shared compute. Control-plane-as-a-service, as in vCluster, solves CRD conflicts by giving each tenant a dedicated API server. But tenants still share worker nodes, leaving compute-level security boundaries unresolved.
Four Open-Source Building Blocks
KubeFlex, KubeVirt, K3s, and OVN-Kubernetes are the four CNCF open-source projects in the stack. KubeFlex is a KubeStellar sandbox project that provisions control planes as a service, supporting K3s, vCluster, and others. KubeVirt lets virtual machines run as Kubernetes pods alongside containers in the same cluster. K3s is a fully conformant Kubernetes distribution packaged as a single binary under 100 megabytes, serving as the per-tenant control plane. OVN-Kubernetes is a CNI plugin using Open Virtual Network and Open vSwitch to deliver software-defined, per-tenant networking.
Per-Tenant Architecture: Control Planes, VMs, and Dual Networks
A hosting Kubernetes cluster runs KubeFlex, KubeVirt, and OVN-Kubernetes. For each tenant, KubeFlex provisions a K3s control plane running in its own namespace. KubeVirt creates VMs that become the tenant’s worker nodes; K3s agents inside each VM connect back to that control plane. Each tenant also gets two user-defined networks (UDNs): one for data plane traffic, one for control plane traffic. No traffic from the default network can reach any tenant, and tenants cannot reach each other. Hardware-level VM boundaries enforce the data plane separation without requiring dedicated physical nodes.
Latency Overhead Stays in the Millisecond Range
To measure the cost of three isolation layers, the team ran LLMD with inference-perf as the load generator. Three load stages: 1, 5, and 10 requests per second, generating 360 prompts total. Median overhead falls in the single-digit millisecond range. CPU and memory overhead of the KubeVirt VMs was measured separately to keep the latency results clean. Next steps include testing with real GPU inferencing and larger models, and integrating this isolation pattern into the KubeStellar console.
Applying the Pattern to AI Agent Sandboxing
Paolo used OpenClaw as a concrete example. OpenClaw runs agent commands directly on the host, and sandboxing is off by default. Plugins can get full host access. Running this kind of workload inside the framework changes that: the agent runs inside a KubeVirt VM, making the host unreachable. A per-tenant API server limits the blast radius. UDN segments the network traffic. The pattern doesn’t cover everything, but it addresses the core containment problem for untrusted AI agents on shared Kubernetes infrastructure.
Q&A
How do you prevent the noisy neighbor problem when KubeVirt VMs are still pods on shared nodes? Tenant workloads run inside VMs, so resource consumption is bounded to whatever CPU and memory the cluster admin pre-assigned to those VMs. ▶ 29:27
The demo shows one control plane per tenant. How do you handle resilience if it crashes? Kubernetes restarts the pod, causing brief downtime. KubeFlex can provision multiple control plane replicas for HA; the single-plane demo was kept simple for clarity. ▶ 31:02
Which Kubernetes distribution and OS does the top-level host cluster use? Vanilla Kubernetes on Ubuntu. The team deliberately avoided dependencies on a specific OS or distribution so the pattern works on any conformant cluster. ▶ 33:40
Notable Quotes
There is always the noisy enable problem Paolo Dettori · ▶ 02:54
Our approach targets three design goals. Braulio Dumba · ▶ 05:44
here’s the problem open CL actually runs Paolo Dettori · ▶ 26:22
overhead is minimum. So we are talking Paolo Dettori · ▶ 27:49
Key Takeaways
- Each tenant gets a dedicated K3s control plane, KubeVirt VMs, and two user-defined networks on shared hardware.
- Latency overhead from three isolation layers measures in the single-digit millisecond range at 10 requests per second.
- The same pattern contains AI agents running untrusted plugins by making the underlying host unreachable at the VM level.
About the Speaker(s)
Paolo Dettori is a Sr. Technical Staff Member at IBM Research in New York. His research focuses on distributed systems, cloud, and open-source technologies. He has authored and co-authored several journal and conference papers and holds multiple patents issued in the United States.
Dr. Braulio Dumba is a Staff Research Scientist at IBM Research. He joined IBM in 2018 under the Hybrid Cloud organization. His current research centers on edge computing and hybrid cloud computing. Dr. Dumba earned a Ph.D. in Computer Science from the University of Minnesota.