AI
Why Traditional Prompt Injection Has Limits
Early prompt injection tricked standalone chat models with phrases like “ignore previous instructions” or “act as an unrestricted catgirl.” Once LLMs gained tools (browsers, code editors, MCP servers), those tricks gained teeth. A poisoned RAG source or a malicious email now reaches agents that can write files and execute shell commands.
Hand-crafted injections are context-dependent and not reusable. Each new target requires fresh analysis. Attackers who need precise output, a shell script where every character matters, had no reliable method. That gap is what this research set out to close.
Universal Adversarial Triggers: One String, Any Payload
The attack splits into two reusable parts: a trigger and a payload. The trigger is a short sequence of mostly unreadable tokens. Wrap any payload (plain text, JSON, XML, a shell command) between a trigger prefix and suffix, and the model ignores its original instructions and outputs the payload verbatim. The same trigger works across different applications and inject locations, as long as the target model stays the same.
“a single magic string can reliably force” — Jiashuo Liang
Liang’s team trained triggers on Qwen 2.5-7B-Instruct, Llama, and DeepSeek Small. Across more than 10,000 diverse conversations,
“experiment, it achieves about 70%” — Jiashuo Liang
success rate across tasks. Once someone obtains a trigger, inserting a new payload requires no retraining.
Remote Code Execution on AI Agents
Open Interpreter reads a malicious email whose shell command is wrapped in a trigger prefix and suffix. The trigger forces the model to output the command in the exact format the agent expects. A reverse shell lands in the attacker’s Docker container. Open Interpreter RCE via malicious email (10:19)
The Cline demo poisons an MCP tool description. The attacker embeds the trigger and an XML payload with requires_approval set to false. When a developer asks Cline to describe the project, Cline executes the shell command on the VS Code terminal with no confirmation prompt, and a calculator opens. Cline VS Code extension RCE via poisoned MCP tool (13:05)
Gradient-Based Optimization: Training the Trigger
GCG (Greedy Coordinate Gradient) finds trigger tokens by adapting HotFlip (2018), which estimates how the loss changes when one token replaces another in the embedding space. Each iteration samples random token positions, picks top-k replacement candidates, tests actual loss, and keeps the best swap.
Training required over 10,000 diverse conversations and roughly 100,000 LLM invocations to reach 70% attack success. Triggers transfer within families (Llama 3.1-8B to 70B, Qwen 2-7B to 2.5-7B, both around 60%) but not across families. White-box model access is required, and perplexity-based filters can detect the gibberish token sequences.
Q&A
Is the trigger dependent on the specific payload it was trained with, or can you swap in a different payload afterwards? The trigger is designed to be payload-independent, trained across diverse payload-context pairs so the same trigger string works when any new payload is substituted in. ▶ 22:48
Could triggers ever generalize across model families, for example by training on multiple models simultaneously? The speaker did not test it but suggested training triggers on multiple models simultaneously or independently and combining them could eventually generalize to proprietary models. ▶ 24:48
Notable Quotes
a single magic string can reliably force Jiashuo Liang · ▶ 0:23
experiment, it achieves about 70% Jiashuo Liang · ▶ 8:48
are not trustworthy by default. We Jiashuo Liang · ▶ 22:04
Key Takeaways
- A single trigger string achieves ~70% attack success across diverse prompt contexts and payload types.
- Triggers require white-box model access to train but need no modification at attack time.
- AI agents should always run sandboxed with explicit tool-execution controls to limit prompt injection impact.