Manual Review: The Starting Point

▶ Watch (4:12)

Daniel added a get by ID endpoint to a products API. The code had function-level access control using a read claim. Tobias reviewed the pull request. He asked who can get a valid token. The access model revealed a multi-tenancy system with a market concept. Each user should only see products for their market. The market check was missing. Tobias called it a critical security issue. The team fixed the code and added negative tests to prove authorization works.

AI Without Context Misses the Bug

▶ Watch (19:04)

The same pull request was reviewed by the GitHub Copilot review agent. The agent complained about indentation, inconsistent patterns, and missing unit tests. It said nothing about broken access control. From a security perspective, the review was useless. It missed the BOLA completely. The agent knew nothing about the access model. Without that context, it could not find the bug.

Adding Context Makes the Agent Effective

▶ Watch (20:16)

Daniel added a one-line access model description to the co-pilot instructions file. The agent ran again. Its first comment stated the get by ID method does not verify the user’s permission to access the product’s market. It even suggested using the permission service. The bug was caught. Clear patterns like domain primitives and a permission service made detection easier. Later, with full instructions, the agent also spotted an SSRF risk in a webhook endpoint.

From Security to Compliance with MCP

▶ Watch (31:57)

To reduce hallucinations, Sandberg and Ahnoff built an MCP server with separate RAG indexes for security and compliance. One index held OWASP resources and blog posts. The other held only the internal CIS controls library. The agent called security QA and compliance QA functions. The review now included references to specific security controls and ASVS items. This bridged the gap between compliance policies and daily DevSecOps work.

Q&A

How do you scale this across teams? Use a centralized repo for instruction files, push them out, and let each team add their own access model. ▶ Watch (45:20)

Can the MCP server be called as an API without an agent? Yes, it sits behind an Azure function; you can call the endpoint directly or put it in a web page as a questionnaire. ▶ Watch (48:45)

Did you try using multiple models in parallel to reduce hallucinations? Not in this demo, but in production-like environments it can work. Models could vote, and the minority vote often catches stumbles. Humans still need to verify. ▶ Watch (49:35)

Notable Quotes

who can get a valid token for this API? Tobias Ahnoff · ▶ Watch (5:57)

we have a critical security issue here we identified. Tobias Ahnoff · ▶ Watch (6:43)

it found it. Tobias Ahnoff · ▶ Watch (21:06)

it’s not deterministic in that way. Daniel Sandberg · ▶ Watch (40:31)

Key Takeaways

  • Eight security review questions (auth, input, dependencies, secrets, business logic, logging, tests) capture the most important aspects quickly.
  • AI agents find security bugs reliably when given explicit access model and architectural context.
  • Humans remain mandatory in the loop because AI hallucinates and compliance may require separation of duties.