AI
Why 1,000-Node Clusters Break Storage Bandwidth
A Kubernetes cluster with 1,000 nodes hitting the same file or image simultaneously generates 1,000 concurrent download requests. Storage bandwidth hits its ceiling regardless of how much you provision, because every backend storage system has a hard limit. Dragonfly answers with two strategies: P2P transfer that draws on idle node bandwidth to absorb the load, and chunk-level deduplication plus on-demand downloading to shrink the total bytes transferred. Nydus, a Dragonfly sub-project built on the RAFS format, cuts container image launch time from minutes to seconds using on-demand pulls.
The 2.4.0 Protocol and Scheduling Improvements
Dragonfly 2.4.0 replaces gRPC between peers with a TLV (type-length-value) binary protocol. Over TCP, large-file download time drops by half. Over QUIC, it drops 40%. Scheduling now scores candidate parents on three metrics: peak bandwidth usage, bandwidth duration ratio, and concurrent efficiency. Peers connect to multiple parents simultaneously and pick the best one piece by piece. When two peers pull from a single seed, they download interleaved pieces and swap what each is missing, which cuts root-peer overload and reduces long-tail download times.
Preheating and Cache Distribution Controls
Preheating pushes files into peers before they are needed, removing download latency from the critical path. Version 2.4.0 lets operators specify preheating scope by IP percentage or absolute node count, making it practical to warm a targeted fraction of a large fleet. A new rate limiter caps how many peers pull from the origin concurrently during preheat, preventing origin overload. Dragonfly also now calculates task IDs from a hash of image blocks rather than the download URL, which prevents redundant downloads when the same block appears in multiple artifacts.
Packaging and Managing AI Models as OCI Artifacts
AI model weights have grown from hundreds of megabytes to hundreds of gigabytes and into terabytes. Most teams today manage them through custom object-storage scripts or Git LFS, outside any standard artifact pipeline. Dragonfly’s answer maps the existing container delivery workflow onto models. A CLI tool called Model Control packages model weights, configuration, and metadata into an immutable OCI artifact. Harbor stores and versions that artifact, enforces RBAC, applies retention policies that delete inactive versions, integrates Cosign and Notation for signing, and replicates between central and edge registries.
Delivering Models to Inference Nodes with P2P and Volumes
In the delivery stage, Dragonfly P2P distribution uses 80% of each node’s idle bandwidth. For latency-sensitive inference, Harbor triggers Dragonfly to preheat and cache the model on target nodes before the service scales. When the inference engine starts, the model loads from local disk. The engine runs as a container and the model mounts as a separate OCI volume, letting multiple pods on the same node share the same model files. OCI volume support shipped as alpha in Kubernetes 1.31 and promoted to beta in Kubernetes 1.33. A model CSI driver covers older Kubernetes versions.
Live Demo: Model Control to Harbor to Inference Pod
The demo pulls a Hugging Face model, uses Model Control to generate a Modelfile (analogous to a Dockerfile), builds the model into an OCI artifact, and pushes it to Harbor. Harbor displays the model’s tag, annotations, and summary. A Dragonfly P2P provider preheats the model to a node. A Kubernetes pod definition lists the model as a volume and the inference engine as the container image. CRI-O downloads both, and the model artifact mounts into the running inference pod. The model and the engine stay fully separate artifacts throughout.
Notable Quotes
Dronely delivers efficient, stable and secure date distribution and acceleration powered by the P2P technology. Wenbo Qi · ▶ 00:23
use the TCP based vertex the large fail download time is reduced by half compared to the gRPC. Wenbo Qi · ▶ 07:51
It will use the 80% of each nodes idle bandwidth and improving the model deployment efficient Wenbo Qi · ▶ 18:41
I think this solution uh SP separ separates the model from the engine. Wenbo Qi · ▶ 27:32
Key Takeaways
- TLV protocol over TCP cuts large-file download time by 50% versus gRPC.
- Dragonfly P2P uses 80% of idle node bandwidth for AI model distribution.
- OCI volumes for models reached beta in Kubernetes 1.33, separating model from engine.
About the Speakers
Wenbo Qi is a software engineer at Ant Group and a maintainer of both Dragonfly and ModelPack. He contributes to open source software and holds that fear springs from ignorance.
Chenyu Zhang is a software engineer at Ant Group focused on the CNCF projects Harbor and Dragonfly, with additional experience in DevOps and cloud-native technologies.