AI
The Trust Gap and Agent Risk Taxonomy
The University of Melbourne and KPMG surveyed 50,000 people worldwide and found trust in AI falling even as usage climbs. 81% of leaders plan to deploy agents alongside their workforce in the next 12 to 18 months. 85% are worried about prompt injection attacks. 77% are concerned about agents drifting off task.
The International AI Safety Report divides AI risk into malfunctions, misuses, and systemic impacts. Bird focuses on the first two: agents hallucinating or getting hijacked. Fixing those, she argues, requires developers, security teams, and compliance officers collaborating through planning, testing, and deployment rather than handing off at the end.
Model Selection and Guardrail Configuration
Foundry’s model leaderboard includes safety benchmarks. Bird checks attack success rates, notes GPT-4o mini defends against the most attacks, then chooses GPT-4o for the demo. She layers three guardrail types: content filters, Prompt Shields extended for indirect attacks through tool calls and data, and a task adherence guardrail that blocks or escalates when the agent drifts. In the Foundry playground safety scoring (12:34), each response gets a real-time intent score and safety risk rating.
A spotlighting feature, also new at Build, marks tool-returned data as untrusted before the model processes it, helping Prompt Shields separate attacker input from legitimate requests.
Agent Identity, Inventory, and Data Governance
Every agent built in Azure AI Foundry now gets an Entra Agent ID automatically. Haiby opens the Entra admin center, filters by agent type, and shows all Copilot Studio and Azure AI agents in a single inventory. Security teams can revoke or delete agents from that view without developer involvement.
Purview policies also apply org-wide once a developer drops the Purview SDK into their agent. A single detect-sensitive-info policy covers social security numbers, credit cards, and passports across every agent in the organization with the integration enabled.
Evaluation: Intent Resolution, Tool Accuracy, and Adversarial Testing
Foundry’s new agent evaluators measure three properties standard quality evals miss: intent resolution, tool call accuracy, and task adherence. Running the Contoso Outdoors agent returned 83% on intent resolution, 70% on task adherence, and 60% on tool accuracy. Those gaps tell the developer exactly where to iterate.
Sameki shows how governance platforms (Credo AI and Saidot) map registered risks directly to Foundry evaluation plans and generate adversarial test data automatically. She walked through the red teaming report showing 11% attack success (30:10), where Base64-encoded prompts were the main vector getting through.
Runtime Monitoring and Threat Detection
The same evaluators run as continuous evaluators post-deployment, feeding a Foundry Observability dashboard that tracks task adherence and relevance per call. When Bird spots a jailbreak alert, Haiby walks through the Defender incident view (37:17): a full incident record showing the suspicious prompt segment, MITRE ATT&CK mapping, ASCII smuggling (an indirect prompt injection), and a user phishing attempt.
Defender now covers more than 15 generative-AI-specific detection types, including wallet attacks and jailbreaks. Security teams can block or suspend the offending user directly from the incident.
Commerzbank Ava: Content Safety in a Production Banking Agent
Commerzbank went live with Ava one month before Microsoft Build 2025. Ava handles general banking questions, account-specific queries like interest rates, and end-to-end transactions including credit card limit changes and card blocking, secured via photoTAN as a second factor.
Mosner describes the content safety stack: prompt engineering first, then five harm categories (jailbreak, hate, sexual, violence, self-harm) plus custom categories for roleplay and manipulation, with thresholds tuned per output type. Requests outside scope route to the LLM or get blocked. Sensitive topics hand off to a 24/7 advisory center via agent.
Notable Quotes
85% are worried about prompt injection attacks. Sarah Bird · ▶ 5:31
Banking Avatar is our lighthouse project for AI. Gerald Mosner · ▶ 41:46
It’s just that agents makes it feel more urgent. Sarah Bird · ▶ 48:20
Key Takeaways
- Entra Agent ID automatically assigns identity to every Foundry agent, giving security teams a full org-wide inventory.
- Prompt Shields, spotlighting, and task adherence guardrails extend existing defenses to the indirect attack surfaces agents introduce.
- Adversarial red teaming integrated via governance platforms like Credo AI generates attack data developers lack on their own.