AI
The Problem: Six Tickets, Five Teams, No Visibility
Before the blueprint, deploying a new service at Saxo Bank meant opening six separate tickets or emails across five infrastructure teams: a server, firewall rules, a service account, an SQL role grant, DNS, and a TLS certificate. Each request sat in an invisible queue. Developers had no way to check whether the AD ticket had been picked up or whether the DBA had started on the role. If anything failed, the whole process restarted from scratch. There was no authoritative record of what should exist.
One File, One PR, One Approval
A developer now writes a single YAML file declaring the service name, short identifier, which Kafka topics it consumes, which APIs it provides, and who owns it. They open a PR. The verifier service auto-assigns reviewers based on what the change touches: role provisioning pulls in the teams responsible for those roles. Services start with lifecycle set to “implementing” for fast dev and test access, then request a “production” lifecycle change that pulls in security, GCC, and governance reviewers before anything reaches live. The file requires no knowledge of AD syntax or Kafka ACL format.
How a YAML File Becomes Infrastructure: Blueprint Engine and Operators
The blueprint engine runs inside Kubernetes, polls the service catalog Git repository on an interval, applies environment overlays, and converts each service definition into a blueprint custom resource. The blueprint operator then reconciles those CRs through per-domain transformers that emit child CRs: namespace requests, gMSA entities, Kafka ACL requests, and more. Specialist operators watch each resource type and drive the actual systems. The catalog holds more than 1,000 services across 45 business domain folders, with one file per service driving dev, test, sim, and live environments.
Handling Windows Servers and Legacy Workloads
Some Saxo services have run for 15 to 20 years on Windows servers. Containerizing them carries cost and risk that produces no business value, and some use IP multicast that the current Kubernetes network setup does not support cleanly. For these, the gMSA transformer reads the service short name, cross-references the infrastructure catalog to find the server collection, and emits a gMSA entity CR. The AD operator calls a constrained PowerShell remoting endpoint over WinRM with Kerberos authentication. The operator never touches the domain controller directly. Security segmentation holds at the constrained endpoint boundary.
Disaster Recovery and Team Scale
Recovery means standing up a fresh cluster and pointing it at the same Git repositories. The blueprint engine picks up on the next poll cycle, regenerates all CRs, and the operators reconcile AD, Kafka, Azure, namespace, and policy repositories back to desired state within one cycle. This matters for compliance: after a potential intrusion, auto-provisioning access controls and network policies into a clean environment makes recovery realistic rather than theoretical. The platform team runs with three or four full-time members and integrates through interfaces maintained by expert teams, avoiding operational ownership of AD, Kafka, Azure SQL, or Elastic.
Developer Visibility Through Backstage and an Embedded AI Assistant
Service definitions follow the Backstage catalog format, so all 1,000-plus components are automatically ingested into Saxo’s internal Backstage instance, called The Reach. Developers can inspect CR provisioning status, AD group creation, and dependency graphs directly from their service page. An embedded AI assistant named Eva reads the service catalog, queries blueprint CRs, and runs limited kubectl and Prometheus lookups. It can identify the current Backstage page, suggest next steps, and check whether a specific firewall opening is live, all without access to secrets.
Notable Quotes
there’s no authoritative record of what should exist. There just is what exists. Oskar Kristiansen · ▶ 03:13
The developer owns the what, and the platform owns the how. Oskar Kristiansen · ▶ 05:54
once we had the shared framework, the core reconciliation loop, the status management, etc., then it’s actually quite a small focused piece of code to add a new operator. Oskar Kristiansen · ▶ 16:55
We have learned the hard way that depending on transitive properties like, for example, the status on the custom resources to determine if a change should be made or not can cause unforeseen consequences Oskar Kristiansen · ▶ 17:51
Key Takeaways
- One YAML file and one PR replace six manual tickets across five infrastructure teams.
- Custom operators beat Crossplane and Terraform for internal APIs with no existing providers.
- A three-person team covers AD, Kafka, Azure, and Elastic by integrating through interfaces those expert teams already maintain.
About the Speaker(s)
Oskar Kristiansen is a Senior Enterprise Platform Engineer at Saxo Bank. He describes himself as passionate about Cloud Native open source technologies and their application in business, and holds the Kubestronaut certification.