AI
Three Conditions That Define Indirect Prompt Injection
Indirect prompt injection has three components. First, malicious instructions must be embedded in an external data source: documents, emails, web pages, or database records. Second, the AI system retrieves and processes that data as part of inference, typically through a tool call. Third, the system takes a dangerous action without human confirmation. Vandevanter argued that the first two conditions alone represent risk. Only when the third condition is met does the risk become a vulnerability requiring a conversation about impact.
Mapping Tools and Approval Levels Before Testing Exploits
The most useful first step is building a table of every tool the AI agent can call, its capabilities, and the approval level for each action. A calendar assistant might have tools to create contacts, send emails, and modify entries. For each tool, the question is whether a human must approve before the action executes. Approval logic in the system prompt is immediately vulnerable: prompt injection can suppress the approval check. If the approval logic lives in a config file, a file write tool can overwrite it and bypass the check entirely.
Memory Poisoning and Code Execution as AI-Specific Attack Surfaces
Memory tools let agents persist state between conversations. That persistence is also an adversary foothold. Through indirect prompt injection, an attacker can write a memory entry that instructs the agent to forward file contents to an external destination whenever a specific file type is opened. Vandevanter confirmed this works against agentic IDEs. File-based memories carry an additional bypass: even if the memory tool requires approval, a file write tool can overwrite the same file. A file hash and signature on each memory entry blocks that route.
Tool Chaining and the Exfiltration Step That Completes Every Attack
Every indirect prompt injection attack Vandevanter has analyzed includes an exfiltration step. One tool reads data (a file, a database record) and a second writes it out (an image markdown render, an outbound web request). Microsoft’s blog post on defending against indirect prompt injection lists four known exfiltration techniques as of August. Breaking the chain at any link stops the attack. Labeling each tool call’s data source and flagging unsafe sequences such as read-filesystem then write-markdown-image gives defenders a signal similar to EDR process chain analysis.
Six-Stage Defense in Depth for Deployed AI Agents
Input moves through six stages before reaching the LLM and again on the way out. Input sanitization uses regex to filter invisible Unicode characters. Input guardrails can run in non-blocking mode (similar to a WAF), so teams detect violations before deciding to block. Context validation rejects requests that fall outside the agent’s intended scope. An output guardrail catches PII and successful jailbreak responses. Output sanitization strips markdown exfiltration payloads. Human-in-the-loop exits are available at each stage. Agents that repeatedly violate these stages can be suspended for review.
Automating Recon and Fuzzing with TamperMonkey
AI model responses are probabilistic. An exploit may fail five times and succeed on the sixth. Manually retyping test cases does not scale. Vandevanter built a TamperMonkey browser extension script that connects via WebSocket to a server running the full recon playbook. The script writes prompts into the agent, clears chat history between runs, and saves every response. It supports multiple simultaneous clients for parallel fuzzing. He ran it live against Google Slides to enumerate Gemini tools. Four demo labs released with the talk include the automation script and use local Llama inference.
Q&A
How does NeMo Guardrails compare to AWS Bedrock guardrails and Llama Firewall? Vandevanter likes NeMo’s guidance and finds its recommendations clear, but has not tested the alternatives head-to-head and could not offer a comparison. ▶ 34:59
Are frontier reasoning models meaningfully more resistant to prompt injection? Yes, roughly 15% more resistant based on the LERA breaker benchmark; open-weight models are closing the gap, and deployment architecture matters more than model choice. ▶ 36:49
Are AI agents introducing genuinely new attack classes or mainly enabling old ones? Mostly old techniques shipped faster; multimodal injection via images embedded in documents is the clearest new surface, though it still ends up being prompt injection at execution time. ▶ 38:37
Notable Quotes
almost always an exfiltration component Will Vandevanter · ▶ 19:14
revealing ABSAC gaps in certain ways. Will Vandevanter · ▶ 11:57
probabilistic, not deterministic. That’s Will Vandevanter · ▶ 28:01
Key Takeaways
- Indirect prompt injection only becomes a vulnerability when it triggers action without human confirmation.
- Build a tool capability table with approval levels before writing a single test case.
- Chain tool calls almost always include an exfiltration step; breaking that link stops the attack.
About the Speaker(s)
Will Vandevanter is a Security Engineer on the AI/ML team at Trail of Bits, where he performs security audits across all layers of AI/ML infrastructure. He has previously spoken at Black Hat, DEF CON, and OWASP conferences and has released open source security tooling.