AI
Volcano’s Evolution Into a Unified Scheduling Platform
Volcano started as the kube-batch project. It joined CNCF in 2020, reached incubating status two years later, and is targeting graduation in 2026. The scheduler handles TensorFlow, MPI, and Spark jobs, manages multi-level queues, and supports heterogeneous devices including GPUs, NPUs, and TPUs. In 2023 the workload mix shifted. Massive distributed LLM training appeared first, then disaggregated prefill/decode inference patterns, and now short-lived bursty agentic workloads with strict latency requirements. Each type demands different scheduler behavior.
Two Schedulers, One Cluster: The Agentuler
Volcano v0.14 introduced the agentuler alongside the existing batch scheduler. The batch scheduler uses a session-based mechanism that fires every second by default, which is too slow for serverless-style agentic traffic. The agentuler runs multiple parallel workers each holding a cluster snapshot, uses optimistic concurrency with conflict-aware binding, and shares the predicate, node-order, and bin-pack plugins from the batch scheduler. A new NodeShard CRD acts as the contract between the two schedulers, dynamically assigning node pools so the two never conflict over the same nodes.
Dynamic Node Sharding and Workload Collocation
The sharding manager partitions the cluster both vertically (fixed node boundaries) and horizontally (by utilization threshold). One concrete policy targets 80% CPU utilization for the batch scheduler. Above that threshold the agentuler takes over, placing agentic workloads on the fragmented resources that batch training jobs leave idle. A separate collocation feature adds three QoS tiers for online workloads and lets offline best-effort jobs consume up to 30% of node capacity during off-peak hours. A per-node SLO agent monitors actual CPU and memory use and triggers scheduling decisions in real time.
Network Topology Aware Scheduling With HyperNode
The v0.12 release introduced HyperNode, a CRD that models network topology as a tree. Leaf HyperNodes (s0, s1, s2, s3) contain physical node sets. Non-leaf HyperNodes represent higher network tiers. Volcano can now bin-pack based on this tree, scheduling prefill pods onto a leaf topology and decode pods across a wider tier-2 topology for disaggregated LLM serving. Auto-discovery translates NVLink Fabric Manager, InfiniBand, and RoCE LLDP data into HyperNode labels automatically. The team is pushing HyperNode as a Kubernetes first-class citizen through the CNCF, since other schedulers still rely on raw node labels.
Casina, AgentCube, and Volcano Global
Three sub-projects extend Volcano beyond scheduling. Casina is a Kubernetes-native LLM serving platform that auto-scales on token throughput, routes traffic with KV-cache-aware and prefill/decode-aware load balancing, and scales prefill and decode instances independently. AgentCube, released in 2024, runs AI agent workloads in sandboxes built on the Kubernetes sidecar sandbox. It provides warm pool reuse to cut cold-start latency and ships an out-of-box Python SDK for session management. Volcano Global adds multi-cluster federation via Karmada, with HyperJob splitting large training jobs across heterogeneous clusters and data-dependency-aware scheduling to cut cross-region IO latency.
Q&A
How do two schedulers share nodes without conflicting? The NodeShard CRD gives each scheduler a node pool dynamically; above 80% CPU utilization the agentuler schedules pods, below that the batch scheduler runs, and both read the same CRD contract to avoid overlap. ▶ 26:59
What distinguishes Volcano from Kueue? Volcano originated from kube-batch, shipped gang scheduling and queue management earlier, and is now extending beyond batch to inference, agent workloads, and reinforcement learning workloads, with heterogeneous device pooling via projects like HAMi for GPU slicing. ▶ 29:13
Notable Quotes
agentic workloads uh came into the picture which are short-lived latency sensitive and extremely bursty Hajnal Máté · ▶ 05:24
rather than an inefficient static partitioning for example when you have when you can when you split the cluster between uh with with node labels or something like that. This is a dynamic mechanism Hajnal Máté · ▶ 09:54
every other uh scheduler like the KO scheduler, the Quler, coordinator scheduler. Um these are handling network topologies based on node labels and uh it’s really cumbersome for administrators to to handle all all these lab labels and it’s just not not the right abstraction. Hajnal Máté · ▶ 18:18
the perfu and decode has a different resource requirement. uh at the prefuel can do uh needs more computer resource but the decode needs a more uh memory bandwidth. Chen Zicong · ▶ 20:27
Key Takeaways
- Volcano v0.14 ships the agentuler, a parallel-worker scheduler built for sub-second latency agentic workloads.
- The NodeShard CRD dynamically splits node pools between the batch and agent schedulers at a configurable CPU utilization threshold.
- HyperNode auto-discovers NVLink, InfiniBand, and RoCE topology and bin-packs prefill/decode pods to matching network tiers.
About the Speaker(s)
Zicong Chen (Jesse) is a top-2 contributor to CNCF Volcano and an R&D engineer at Huawei Cloud focused on cloud-native scheduling for complex AI workloads.
Mate Hajnal is a Senior MLOps Engineer at Aumovio. Before the spin-off from Continental he held DevOps and Platform Engineering roles at Red Hat and Nokia, and recently became a Volcano contributor working alongside the Huawei team on several new features.