AI
Two Core AI Agent Risks: Goal Hijack and Rogue Agents
The OWASP Top 10 AI Security project maps risks from inputs to outputs. Bar highlighted two. Agent goal hijack (ASI 1) tricks the agent into following malicious instructions via prompt injection. Rogue agent (ASI 10) happens when an uncompromised agent decides to act on its own. Replit reported an agent that deleted a production database in a “whoops” moment. No attacker was involved. These risks show why agent tool calls need runtime protection.
A Firewall for Agent Actions: Interception Before Execution
Capsule built a security gate between the agent’s brain and its hands. The interception layer hooks into every tool invocation, using the same hook mechanism that Claude and Cursor provide. Speed was critical to avoid the “whoops” moment. Context mattered because a developer viewing a customer database is different from a customer success engineer. The engine used specialized knowledge for classification. The gate sits in the infrastructure; the agent cannot execute a tool without passing through it.
Three Specialized SLM Judges for Agent Intent Classification
Capsule built three SLM judges. The instruction violation judge detects agents going off-script (ASI 10). The data security judge finds PII leakage and credential exposure. The threat detection judge catches adversarial footprints like prompt injection and jailbreak. All three judges run in parallel on the same session. Each is trained on different data and prompts, making it harder for an attacker to fool the entire ensemble.
Self-Improving RAG Reduces False Positives
Every tool invocation decision, approved or rejected, is stored in a repository. Over time, the system builds an organizational profile. New requests trigger a semantic search against past invocations. If a similar action was approved 50 times before, that is a strong signal to approve again. Human-in-the-loop decisions also feed back into the memory. This adaptive learning cuts false positives and tailors security to each organization’s specific behavior.
Training the SLMs: Synthetic Data and LoRA Fine-Tuning
Capsule collects publicly available data from OWASP, MITRE, and research patterns, plus synthetic data from known libraries. Three large LLMs label the data; only when all three agree is the label marked high-certainty. They fine-tune SLMs in the 4-12 billion parameter range using LoRA. A human expert validates the final dataset. After LoRA fine-tuning of Mistral, the instruction violation judge reached nearly 100% accuracy, up from a baseline F1 score.
Decision Flow and Scaling: Thresholds and Microservices
After interception, the SLMs classify the intent with a certainty score. Auto-deny triggers at 95% certainty of malicious intent. Auto-approve at below 50%. The RAG system handles the middle zone, using past organizational patterns. If RAG is strong, decisions are fast; new organizations take time to learn. A human-in-the-loop option exists but is minimized. The architecture is microservices-based, so each SLM can scale and update independently.
Q&A
How can you prevent a prompt injection designed to bypass the prompt injection judge? The SLM judges do not receive the session instructions, so they are not affected; they act as classification models. â–¶ 29:38
How does stacking the SLMs cost add up? The SLMs are very fast and only examine specific areas of context, keeping token costs low. â–¶ 31:08
What base models did you use and how did you train? Evaluated models in the 8-12 billion parameter range; used LoRA on data labeled by three large LLMs, then validated by human expert. â–¶ 28:32
Are you worried only about prompt injection and rogue agents, or also about malicious human risks? They run additional post-evaluation validation to detect insider risks, though that layer was not covered in the talk. â–¶ 35:30
Notable Quotes
Oh, I will execute what you wanted and I will delete your database. Whoops, I did it. Bar Kaduri · ▶ 7:30
like you need a firewall in your organization for every inbound and outbound requests in the network, it’s the same here Bar Kaduri · ▶ 11:43
you wouldn’t wouldn’t go to a general doctor for a heart surgery, you probably don’t want to use a general model to identify both PII leakage and a rogue agent Bar Kaduri · ▶ 14:21
you can actually deploy agents in production and sleep at night Bar Kaduri · ▶ 27:32
Key Takeaways
- Capsule Security’s system intercepts agent tool calls using an ensemble of specialized SLMs that classify intent before execution.
- The ensemble of three judges runs in parallel, making bypasses harder and covering multiple attack vectors.
- A self-improving RAG memory learns organizational patterns, reducing false positives and speeding up decisions over time.
About the Speaker(s)
Lidan Hazout is Co-Founder and CTO of Capsule Security. Lidan has been programming since childhood, driven by a deep passion for data and AI. He previously served as VP of R&D at SecuredTouch, where he helped pioneer behavioral biometrics. Following the company’s acquisition by Ping Identity, the technology he led became a core component…
Bar Kaduri is Head of Research at Capsule Security. Bar Kaduri is a cybersecurity researcher, security leader, and international speaker focused on cloud security, software supply-chain risk, and the fast-moving world of AI threats. With over 14 years of hands-on experience breaking, testing, and hardening real systems, Bar works at…