AI
What Agents Are Actually Doing in Production Today
Across all five panelists, the pattern is consistent: agents on the read path work, agents on the write path do not ship yet. Log summarization, observability queries, and root cause analysis are live. Autonomous end-to-end platform management is not. AWS built a DevOps agent that correlates logs, traces, and service topology and hands engineers a finished root cause before they open a terminal. The agent does the correlation work; a human still executes the fix.
Agentic SDLC and the Open Source Contribution Problem
Agentic software development covers more than code generation. Quality engineering, documentation, and consistency all change when agents enter the loop. The harder question is whether open source projects accept agent-authored contributions at all. Who triggers a review? What are the quality standards? Cisco ran agentic CI/CD, including both read and write operations, since the end of 2024. That success led Cisco to open-source CAPE (Community AI Platform Engineering) and donate it to the CNCF sandbox the week of the talk.
Non-Determinism Breaks the Reliability Model Platform Teams Built
Platform and SRE teams spent years building testable, observable, deterministic systems. Agents run on non-deterministic models. You cannot model their failure modes the way you model a service crash. Sara Qasmi framed the core challenge: how do you contain non-determinism inside systems that were designed around predictability? Vincent Caldeira’s conclusion from research at Columbia’s financial safety institute lab is that reliability for agentic systems is statistical. Evaluating agents against human performance, not against a 100% pass rate, is the right measure.
Identity, Shadow IT, and the MCP Security Gap
MCP is the first protocol moving from AI teams to platform engineering teams, because platform teams own identity and security. The immediate problem is identity sprawl. An organization may standardize on Okta internally, but each external MCP server, GitHub, Figma, and others, carries its own identity model. Idit Levine put the proportion at close to 100% of customers currently working on this problem. Hasith Kalpage added that without owning both platform and security hats simultaneously, getting agentic capabilities deployed at scale inside a large organization slows considerably.
How to Get Started Without Creating a Mess
Vincent Caldeira’s first two steps: define the use case and the value metric, then inventory the data your current process already produces. Technology choice follows from data, not the reverse. He also called an AI gateway the first architectural decision to make, before anything else, because it provides observability into what agents are doing. Idit Levine pointed to three CNCF projects: Agent Gateway (Linux Foundation), K-Agent for running agents on Kubernetes, and the new skills registry. Sara Qasmi’s starting point: treat agents as untrusted automation, not intelligent teammates, then expand trust incrementally.
Notable Quotes
we are we spend a lot of like a few years building reliable testable and observable systems and we are trying to bring nondetestics components as you said. Uh the problem is that this uh breaks the reliability model we we built. Sara Qasmi · ▶ 20:39
the conclusion of this is reliability for agentic system is statistical. It’s not deterministic. We can only look at it in a statistical manner. Vincent Caldeira · ▶ 24:15
Safety of AI we can solve. We will solve it in 2026. Agent identity, how to be zero trust with agent. Look, we have freaking good engineer here. We have 13,000 engineer in CNCF working on those questions. Vincent Caldeira · ▶ 30:01
stop thinking of AI as a like a intelligent teammate but uh put it under like the same sale guard um uh guard rails as any other uh untrusted automation Sara Qasmi · ▶ 32:55
we will not be replaced ever guys like I mean we are tier zero application. No one will trust agent for that in probably five to 10 years. Idit Levine · ▶ 08:40
Key Takeaways
- Agents work on the read path today; autonomous write-path actions are not yet in production.
- Cisco open-sourced CAPE and donated it to the CNCF sandbox after running agentic CI/CD since late 2024.
- Agent reliability is statistical, not deterministic; evaluate against human performance, not a 100% pass rate.
About the Speaker(s)
Vincent Caldeira is Red Hat’s CTO for Asia Pacific and a visiting scholar at Columbia University, where he works with a financial safety institute lab on agentic AI system safety. A Top 10 APAC CTO in 2023, he brings more than 20 years of finance IT experience and focuses on open source and cloud-native technologies.
Sara Qasmi is a Platform Engineer at NTT Data with a background in software engineering and DevOps, focused on building infrastructure.
Carlos Santana is a Senior Specialist Solutions Architect at AWS leading Container solutions in the Worldwide Application Modernization group. He is a CNCF Ambassador with deep experience in Kubernetes and GitOps.
Hasith Kalpage is Director of Platform Engineering and CISO at Cisco’s incubation unit, where he led the cloud-native overhaul of Webex’s SaaS backend during the COVID-19 response.