AI
The Vicious Cycle: Why Current AI Defenses Have No Guarantees
Shumailov’s team tested every standard AI defense on static benchmarks, then hit them with stronger attacks on a budget under one dollar. All of them broke. The pattern repeats: a weak model scores well on robustness evaluations because it can’t understand the attacks being thrown at it. The model improves. The attack surface widens. A new defense ships. A month later someone breaks it.
“we can’t build defenses because we don’t have good attacks. We don’t have good attacks because we don’t know how good the attacks are because we don’t have good defenses” — Ilia Shumailov
The field has no exit from this loop because today’s defenses carry no guarantees. They are anomaly detectors dressed up as security.
Camel: Data-Instruction Separation with Formal Language Rewriting
Camel separates data from instructions at the architecture level. A user query gets rewritten into a formal language before the model ever sees untrusted content. That frozen program then runs as-is, calling a secondary model only to process external data — and that model can output nothing but a boolean. He walked through Camel customer-support refund eligibility walkthrough (11:10) live: the planner receives a reservation ID, runs eligibility checks through the secondary model, gets booleans back, and navigates a stack of if-statements. An attacker can flip a boolean — nothing more. Control flow stays fixed.
“if the task that you’re actually asking the model can be solved without ever seeing the data, why even show it the data?” — Ilia Shumailov
Task-Data Independence: Pre-Planning Without Seeing the Data
The objection Shumailov hears most: complex agentic tasks are too data-dependent for pre-planning to work. He disagrees. Testing against standard computer-use benchmarks, where the agent’s only tools are click and keypress, his team found that most tasks can be planned in full before touching any live data. The model already knows how to order a PS5 controller on Amazon. It doesn’t need the internet to write the steps.
“if the task that you’re actually asking the model can be solved without ever seeing the data, why even show it the data?” — Ilia Shumailov
The Computer-use agent pre-planning benchmark (16:43) results bear this out. Plans run 3,000 to 4,000 lines of code. By the seventh attempt, the approach solves roughly 80% of tasks at parity with a model that actually browses the web. The only input is the original user query. No data exposure, no prompt injection surface.
Residual Data-Flow Attacks: Cookie Prompt Ad Injection
CFI stops an agent from being redirected. It does not stop the environment from lying to it. Every pre-planned computer-use agent Shumailov tested included a step to detect and dismiss cookie prompts, because European websites require them. That pattern is predictable — and attackable.
He demonstrated Cookie prompt ad-injection attack demo (19:09) live: fake cookie prompts injected into ad slots. The agent’s plan had already committed to clicking “accept” on any cookie dialog it found. Buy the ad space, serve a fake prompt, the agent clicks through. No “ignore previous instructions” needed. The execution flow stays fixed; the attacker just routes it down the branch they want. CFI doesn’t cover it. Data-flow attacks remain a separate problem.
Selmate and the Agent-Environment Trust Layer
CFI stops attackers from hijacking control flow. It does not stop them from feeding the agent bad data inside a permitted branch. Closing that gap requires the environment to cooperate. Shumailov’s team, working with colleagues at UCSD, built Selmate for this: a browser-integrated layer where agents, site owners, and users can declare what the environment is allowed to do during an interaction. Think access control and policy enforcement at the browser level, open source and available now. Pair Camel with Selmate and, in Shumailov’s words, “I promise you you will fail” to break it.
Q&A
Why don’t we see more adoption of standards like Camel — should academic papers push harder on operationalization? The prototype was trivial; the production system took 6 months of code migrations and requires people who understand both AI and security, but their current scheme now deploys as a one-line change and is model-agnostic. ▶ 22:28
What are the hardest security challenges for computer-use agents, and have you seen real attacks against them? Computer-use agents are the worst case because the input is a screenshot and the action is a click, making semantic policing impossible — but running Camel with Selmate eliminates 99.999% of known issues out of the box. ▶ 24:24
Notable Quotes
we’re kind of failing in in every way Ilia Shumailov · ▶ 4:57
we can’t build defenses because we don’t have good attacks. We don’t have good attacks because we don’t know how good the attacks are because we don’t have good defenses Ilia Shumailov · ▶ 5:50
if the task that you’re actually asking the model can be solved without ever seeing the data, why even show it the data? Ilia Shumailov · ▶ 13:09
computer use agents are horrible because you have no semantics. Your input is a screenshot. Your action is I click somewhere. Ilia Shumailov · ▶ 24:56
Key Takeaways
- Stop deploying unprincipled defenses; demand guarantees rooted in data-instruction separation, not anomaly detection.
- Pre-plan agentic tasks without live data access — 80% of benchmark tasks are solvable without ever touching untrusted content.
- Pair CFI-based systems like Camel with environment-level sandboxing like Selmate to neutralize residual data-flow attacks.