AI
Why Infrastructure Promotion Needs Different Treatment Than App Promotion
A bad image tag crashes a pod. Kubernetes recovers it in seconds, Kargo shows a broken-heart icon immediately, and the blast radius is one pod in one namespace. A bad Crossplane claim promotion is different. A real AWS resource changes behind the scenes. RDS migrations can take minutes or hours. A KMS key deletion carries a mandatory AWS waiting period of 7 to 30 days. During that window, a team pays for a resource whose actual state is unknown. Failures are silent. That asymmetry demanded separate promotion treatment.
Sharded Kargo: One UI Across 13 Clusters
A single central Kargo instance managing every cluster would require exposing the Kubernetes API on each one and force engineers to remember which UI covers which environment. RBI’s answer is a sharded, agent-based topology. A central Kargo in the SSD account drives test and prod tiers. Each of the six downstream clusters runs a local Kargo controller that talks only to the nearby Argo CD controller, deployed via the Argo CD operator. Customers map a stage to a cluster with one field: spec.shard. The sharded controller picks up the promotion and syncs locally.
Migrating Crossplane v1 to v2 Without a Maintenance Window
Crossplane v1 created managed resources at cluster scope. Teams in RBI’s namespace-isolated environment could see their claim but nothing beneath it. Crossplane v2 moves managed resources into namespaces, giving teams full visibility in Argo CD, including the KMS key, alias, lifecycle rules, and log bucket that back a single S3 claim. There is no official migration path between the two scopes. RBI built a three-step process: set migration: true on the claim, copy-paste the generated migrationSpec into the new XR, then use Crossplane’s import function to hand ownership back. No cloud resource is touched, so no maintenance window is needed.
LLM-Backed PR Review and Failure Diagnosis
Customers still open a YAML pull request to trigger a migration. Wrong namespace annotations, copy-paste errors, and indentation mistakes are common and do not scale for a platform team to review manually. RBI built a small application with two endpoints. The analyze-risk endpoint runs before promotion: it fetches the PR diff and GitHub metadata, calls Claude 4.6, and returns a risk grade of small, medium, or high that Kargo displays inline. The diagnosis endpoint runs after promotion, but only on Argo CD sync failure. It reads the app name, all resource statuses, and events, then returns remediation steps in structured JSON that Kargo parses and presents to the engineer.
Live Demo: Catching a YAML Indentation Error Before Prod
The demo pushed an XR with migrationSpec placed at the root level instead of the correct nested position. Sharded Kargo created a freight, triggered promotion, and the HTTP step called the internal LLM endpoint. The risk-analysis comment appeared on the GitHub PR. After merging, Argo CD failed to sync. The diagnosis endpoint consumed the app’s resource events and returned the exact path error with a fix hint. The live portion hit a timeout and fell back to a recorded backup, but the structured JSON output in Kargo confirmed the workflow end-to-end.
Notable Quotes
we had wrong assumption at the beginning that all promotions are equal. But they are not. Gabor Horvath · ▶ 07:07
our experience is that the role of the AI is not really a magic it’s just uh a second pair of eyes in the moments when when we when you as a platform operator are under pressure you oversee some complexity it can really help you to identify issues Gabor Horvath · ▶ 33:10
It’s really just a copy and paste. We prepared everything for them that they just need to copy and paste the information and put it into the XR. Ewald Überall · ▶ 17:40
Key Takeaways
- Sharded Kargo maps each stage to a cluster via
spec.shard, keeping one central UI without exposing cluster APIs. - Infrastructure promotions carry silent AWS failure modes that app rollbacks do not; treat them separately with longer soak times.
- A three-step Crossplane v1-to-v2 migration lets tenant teams self-serve without touching live cloud resources.
- Claude 4.6 grades migration PR risk before promotion and diagnoses Argo CD failures after, returning structured JSON Kargo can parse.
About the Speakers
Gabor Horvath is a cloud native specialist with deep expertise in Kubernetes, EKS, and the CNCF ecosystem. As a Senior DevOps Engineer on RBI’s Mercury platform team, he has led the design and delivery of Kubernetes- and cloud-based platforms for the group’s 11 subsidiary banks.
Ewald Überall is a Senior DevOps Engineer at Raiffeisen Bank International. His career spans Siemens AG Österreich from 2000 to 2017, where he moved from small-business IT support through worldwide technical pre-sales to building a video-streaming platform for a German children’s TV channel, during the same period Netflix launched.