AI
Why Device Plugin Falls Short for High-Performance Networking
Telco workloads need SR-IOV devices, CPU pinning, and memory aligned to a single NUMA node. The device plugin allocates each resource independently on the node. A pod can land on a node with available SR-IOV devices yet still fail with an affinity error because the devices span different NUMA domains. DRA moves topology awareness to the scheduler, which sees the full cluster and can enforce single-NUMA and PCI-root constraints before the pod is placed. IBM’s AI training workloads hit the same wall: NIC selection logic that belongs in a scheduler was being written directly into a custom CNI.
One Mega-Driver vs. a Constellation of Smaller Ones
Jouin tried building a single CNI-backed DRA driver and found it unworkable. CNI is over ten years old, has no scheduling integration, and exposes no verb to advertise supported devices. A monolithic driver would need to absorb SR-IOV, macvlan, ipvlan, overlay CNIs, and every IPAM variant. Choochotkaew added that networking is unlike GPU drivers, where one vendor controls the full stack. Every infrastructure differs. The panel agreed a constellation of composable, independent DRA drivers is the better direction, while acknowledging that coordinating multiple drivers that share the same physical device remains an open problem.
Modeling Bandwidth as Consumable Capacity
Two KEPs address bandwidth allocation under DRA. Choochotkaew proposed consumable capacity so that bandwidth on a network device can be reserved at pod creation time rather than partitioned in advance. Scheinkman extended the idea for SR-IOV: a 100 Gbps physical function with 100 virtual functions available is still exhausted once two pods claim 50 Gbps each. His KEP for shared consumable capacity models the PF as the source of record. Both KEPs remain in proposal state. Jouin noted that resource modeling is the largest unsolved challenge in the entire DRA-for-networking effort, with new ideas still arriving.
Migrating Existing Workloads Without Rewriting Deployments
DRA can expose devices as extended resources through the standard API. SR-IOV users can replace the device plugin with the DRA SR-IOV driver and existing workloads continue requesting the same extended resources in their existing deployment specs. No changes to resource claims or claim templates are required. New workloads that need GPU-to-RDMA colocation can then use resource claims to express that constraint. Both modes run in the same cluster at the same time. Scheinkman called this a bridge between what is running today and the new API.
DRA and CNI Are Additive Tracks, Not Competing Replacements
An audience question about Cilium prompted Ojea to clarify the relationship between CNI and DRA. The CNI model, one pod one IP, stays unchanged. DRA is additive: it enables secondary networks, AI multi-network bandwidth, and complex bare metal workloads without breaking the primary network path. Cilium and similar CNIs handle the flat Kubernetes network. DRA handles the special-function layer on top. The panel warned against running a CNI and a DRA driver pointed at the same physical device simultaneously, describing that integration as a long-term concern, not something to attempt today.
Q&A
What DRA drivers can teams deploy today? Ojea’s arnet driver (created at Google, now production-ready with Azure support and Amazon coming) handles RDMA interfaces for AI workflows; the DRA SR-IOV driver exists and is planned to replace the SR-IOV device plugin; a CNI-backed driver was built but is considered too complex to be the final answer; the kubernetes-network-drivers repo Antonio created is open for experimentation. ▶ 20:14
Can SR-IOV virtual functions be attached to pods inside an OpenStack VM acting as a Kubernetes node? Yes, the DRA SR-IOV driver supports this; VFs are moved into the VM and appear as regular devices available for allocation. ▶ 26:58
Could IPAM be broken out as its own separate DRA driver? The idea is interesting and Mike Zappa demonstrated a conceptual version using consumable capacity to model a subnet’s 255 available IPs, but no implementation separates IPAM from the host device driver today. ▶ 24:44
Notable Quotes
it’s it’s it’s not to in my opinion it’s not really a good idea to have a single monolith uh DA driver that will cover all of all all of this different CNIs Lionel Jouin · ▶ 06:43
that’s the whole githubs thing right the githubs work because you just deploy your j and it works everywhere that’s the secret source of kubernetes portability Antonio Ojea · ▶ 19:15
So please don’t use CNI at the same time as the G driver. Antonio Ojea · ▶ 24:37
it mean that we put the thing that scheduler should do in our CNI then we know about the DRA comes and then we see the opportunity is to put the right thing in the right Sunyanan Choochotkaew · ▶ 03:58
Key Takeaways
- DRA moves SR-IOV and NUMA topology decisions to the scheduler, preventing affinity errors at pod startup.
- Two new KEPs model bandwidth as consumable capacity on a shared physical function, not per virtual function.
- Existing device-plugin workloads can migrate to DRA without modifying deployment specs, using extended resource compatibility mode.
- CNI and DRA are parallel tracks; running both against the same physical device simultaneously is unsupported today.
- A single monolithic DRA network driver is the wrong direction; composable, independent drivers are the working group’s consensus.
About the Speaker(s)
Sebastian Scheinkman is a Principal Software Engineer at Red Hat focused on high-performance networking. He maintains the Network Plumbing Working Group and the SR-IOV network operator.
Lionel Jouin is a Red Hat Software Engineer based in Stockholm and a former Ericsson engineer. He contributes to Kubernetes networking with a focus on native secondary network support, including the multinet project.
Sunyanan Choochotkaew is a Staff Research Scientist at IBM Research Tokyo specializing in distributed computing, networking, and performance acceleration. She maintains the Kepler project and contributed the consumable capacity KEP to DRA.
Antonio Ojea is a Software Engineer at Google working on Kubernetes. He is a top contributor to the project, Tech Lead for SIG Network and SIG Testing, and a member of the Kubernetes steering committee.