AI
Argo Workflows 4.0: What Shipped
Argo Workflows is the oldest Argo sub-project. It runs every workflow step inside a Kubernetes pod, letting you define pipelines as steps or a DAG and pass data between those pods. Version 4.0 arrived earlier this year. The headline feature is artifact drivers, a plugin interface for writing custom data transport in and out of pods. Clucas covered this in a separate talk the same day, which will appear on the Argo YouTube channel within weeks.
Full CRD Validation and the Migration Tool
Up through version 3.7, Argo Workflows shipped trimmed-down CRDs. Version 4.0 ships full CRD schemas. A workflow with a typo or a structurally broken spec now fails at submission with a clear error rather than silently at runtime. Version 3.6 introduced plural versions of several primitives. Those have been removed in 4.0. A new CLI command, argo convert, helps users migrate existing workflow definitions to the 4.0 spec. The controller also gained a restart strategy for pods that never started, covering cases where a restart is known to be safe.
Workflow Tracing Already Merged for 4.1
The first feature merged for 4.1 is end-to-end workflow tracing. Argo Workflows is working with the OpenTelemetry SEMCONV group to define what traces at this scale should look like. The feature shows how a workflow spends its time, which steps run long, and where delays accumulate. Traces from inside individual workloads can propagate up and join the single workflow-level trace. GitHub has something comparable, but this implementation is native to Argo Workflows and connects directly to any OpenTelemetry-compatible backend.
Unifying the DAG Engine and Enabling Pod Reuse
Steps and DAGs are currently two separate code paths. That separation produces bugs that appear on one side but not the other. The plan is to implement steps by transforming them into a DAG and running everything through a single DAG engine. That change also opens the door to DAG plugins. Built on top of this, pod reuse would let annotated workflows skip the pod startup penalty. Pod isolation is lost, but for workloads where spin-up time makes Argo Workflows impractical today, this is the only path forward.
Modernising the Expression System
Argo Workflows currently splits expression evaluation across multiple libraries, including one called complate. Keeping these in sync is a maintenance burden. The goal is a single, consistent expression layer where workflow authors can predict what an expression does, test expressions before running a full workflow, and read them without guesswork. A specific friction point today: Argo Workflows’ templating engine conflicts with Helm’s templating system when you want to deploy a workflow via Helm. The new expression system aims to remove that conflict.
Notable Quotes
artifact drivers are probably the biggest bit which is so you can write your own plugins to uh transport data in and out of those pods. Alan Clucas · ▶ 0:43
this will also mean that steps are implemented by transforming them into a DAG and then running them through the DAG engine may lead to DAG plugins in the future. Alan Clucas · ▶ 3:08
the pod spin up time is significant and means that workflows isn’t really the right tool for the job all the time. Alan Clucas · ▶ 3:20
they don’t conflict hopefully with Helm’s templating system which is fun if you want to template a a workflow in Helm as well as using Argo Workflows’ templating engine Alan Clucas · ▶ 4:26
Key Takeaways
- Artifact drivers in 4.0 let teams write custom plugins for pod data transport.
- Full CRD validation in 4.0 surfaces malformed specs at submission, not at runtime.
- A unified DAG engine will eliminate step-vs-DAG bug asymmetry and enable pod reuse.
About the Speaker(s)
Alan Clucas is a Staff Software Engineer at Pipekit and the project lead for Argo Workflows. His background spans software development and DevOps, and he maintains Crumbhole, a collection of open source GitOps tools, in his spare time.