AI
From Batch Training to AI Workloads
Volcano started as kube-batch and became a cloud-native batch system for GPU clusters. Until 2023, most clusters ran batch training. That year, LLM adoption changed the workload mix. Disaggregated inference arrived, where prefill and decode stages have different resource requirements. Agent workloads followed, running in short bursts. Each type makes different demands on the scheduler, and Volcano now targets each from a single control plane.
Extending Volcano to Cover the Full AI Stack
The 2025 road map splits into two layers. At the top, Volcano extends native workload support beyond VCJob to cover LLM training and inference. Agent framework integration arrives via collaboration with external communities. At the bottom, a unified heterogeneous device allocation layer broadens hardware support. Network topology improvements, specifically topology-aware scheduling, aim to improve AI workload throughput. The stated goal is one scheduling platform across all AI workload types.
A Dedicated Scheduler for Agent Workloads
Volcano’s original batch scheduler handles workloads on a per-second basis, too slow for agent tasks that are latency-sensitive and burst-heavy. A new component called the agent scheduler runs alongside the batch scheduler in the same control plane. The two coordinate concurrently. A sharding controller manages a node CRD to coordinate dynamic node port allocation between them. This lets the system handle latency-sensitive agent workloads without disrupting batch jobs running in parallel.
HyperNode and Network Topology Awareness
Large AI clusters lack a unified topology abstraction, which forces operators to manually tune intra-cluster network configuration for maximum performance. Volcano introduces HyperNode, a CRD that models network domains by switch or equivalent performance boundary. The scheduler uses HyperNode to see the actual network architecture and place jobs more precisely. HyperNode also tracks switch congestion status and bandwidth. Discovery is pluggable: Volcano already supports Nvidia Fabric Manager and existing node label discovery, with more backends planned.
Inference Serving and Agent Scheduling Sub-Projects
Casino is a native AI inference serving platform. It supports multiple inference frameworks and autoscales on token throughput rather than request count. An intelligent router handles KV cache prefix-aware routing and prefill-decode group-aware routing. Casino integrates natively with Volcano scheduling and topology-aware placement. AgentCube schedules AI agent workloads and ships an out-of-box SDK so developers can wire custom tools directly into agent workload definitions. Native agent primitives and broader framework integrations are on the road map.
Notable Quotes
we we call for co-creation Zhonghu Xu · ▶ 5:57
Yeah. Yeah. Just three hours. Zhonghu Xu · ▶ 6:20
Key Takeaways
- Disaggregated inference and agent workloads require schedulers built for different latency and burst profiles than batch training.
- HyperNode models switch-level topology as a first-class Kubernetes resource, giving schedulers real congestion and bandwidth data.
- Casino autoscales on token throughput, and AgentCube provides an SDK to build agent workloads that schedule natively on Volcano.
About the Speaker(s)
Zhonghu Xu is a software architect at Huawei Technologies and a core maintainer of Istio, ranking in the top 3 contributors. He serves on the Istio Steering Committee and leads CNCF TAG-Infra, which guides networking projects across the foundation. He also maintains kmesh and several other CNCF projects.