AI
Kubeflow’s Component Map: One Platform, Many Tools
Kubeflow is not a single tool. It ships the Spark Operator, Trainer, KServe, Katib, Pipelines, Model Registry, and Notebooks as components that platform engineers and data scientists can use independently or together. The typical flow runs from data preparation in Spark, through feature engineering and notebook-based experimentation, into automated pipelines for training and deployment. Each component has its own release cadence and is also bundled in the Kubeflow AI Platform manifest repository.
Notebooks 2.0 and the Unified Python SDK
Notebooks 2.0 shifts Kubernetes complexity to platform engineers. Admins define GPU resource templates and control which images and runtimes are available. Data scientists pick from those options without writing any YAML. The unified Python SDK, published at sdk.kflow.org, covers Trainer, Katib, Spark, Model Registry, and soon Pipelines. It runs locally first and connects to a remote cluster later with no code changes. MCP integration is also planned, starting with Trainer, so agents can trigger fine-tuning runs using natural language.
Trainer v2 and Katib: Training Progress on the Control Plane
Trainer v2 replaces per-framework CRDs with a single TrainJob API that works for PyTorch, JAX, and XGBoost. A new contract between the pod runtime and the training controller pushes real-time job progress to the control plane, so kubectl get trainjob shows a live status. Katib plugs into this same API for hyperparameter experiments. Hugging Face Transformers gained a Kubeflow callback so training with Trainer requires zero code changes. Upcoming work includes a training UI with progress bars and PyTorch elastic training triggered by runtime metrics.
KServe: LLM ISVC and Multi-Node Inference
KServe added a dedicated CRD for LLM workloads, the LLM ISVC, shipped as a separate Helm package to avoid conflicts with the existing InferenceService. It supports multi-node inference, prefill-decode disaggregation, and KV cache offloading across CPU and storage tiers. KV cache-aware routing sends each request to the instance most likely to hold the relevant cache, improving time-to-first-token. The gateway layer, demonstrated with Envoy Gateway, adds token-level rate limiting, usage metrics, and MCP route support. Nvidia, AMD, and Intel AMX accelerators are all supported.
KFP: SeaweedFS, Kubernetes CRDs, and What Comes Next
KFP replaced Minio with SeaweedFS as the default object store. Minio switched to AGPL in 2019 and recently entered maintenance mode. SeaweedFS uses Apache 2.0 and solves multi-tenant artifact isolation by restricting access to namespace-specific subpaths within a single bucket. Pipeline definitions now have Kubernetes CRDs, enabling GitOps workflows with kubectl apply. All KFP containers now build as multi-arch manifests for AMD64 and ARM64, covering AWS Graviton and Apple Silicon. Upcoming: replacing the two-pod-per-component model with one additional pod per workflow, dropping MLMD for a native schema, and an MLflow native plugin from Red Hat contributors.
Notable Quotes
so cubeflow provide a single unified Python SDK that gives access to all the cubeflow components and that makes it easy to use all the popular a IML libraries and framework like pytor deep speed jack on kubernetes but it also provide a local execution mode that you can use to get started locally and then you can you can um connect to your remote cluster later on without changing your um your code basically Antonin Stefanutti · ▶ 07:08
in V2, every component in your DAG will actually spawn two pods, which is kind of resource intensive Michael Zazula · ▶ 22:45
seaweed has a more permissive Apache 2.0 license. It’s more well-maintained and provides a much simpler operational model. Above all, it elegantly solves for multi-tenant artifact isolation by restricting access to namespace specific subpaths in the context of a single bucket Alexander Perlman · ▶ 20:27
Key Takeaways
- Trainer v2’s single TrainJob API replaces per-framework CRDs and streams real-time progress to the control plane.
- KServe’s new LLM ISVC CRD enables multi-node, disaggregated inference with KV cache-aware routing.
- KFP replaced Minio with SeaweedFS (Apache 2.0) and now builds multi-arch containers for ARM64 and AMD64.
About the Speakers
Johnu George is a Technical Director at Nutanix leading the AI Systems team. He has driven industry collaborations across Kubeflow, KServe, and Knative, with research interests in machine learning systems design and distributed learning infrastructure.
Antonin Stefanutti is a Principal Software Engineer at Red Hat on the OpenShift AI team, focused on MLOps and distributed model training. He maintains the Kubeflow SDK and the Kubeflow Trainer.
Valentina Rodriguez Sosa is a Principal AI Architect at Red Hat specializing in FSI safety and security. She brings more than 18 years of experience defining and implementing software architectures and serves as a Kubeflow product maintainer.