AI
Operation Pale Fire: Red Teaming Goose
Josiah Peedikayil from Block’s offensive security team led Operation Pale Fire, a red team operation targeting Goose, an early AI agent internal to Block and later open sourced. Goose shipped with a developer shell tool call that ran bash commands. The team wanted to simulate an external attacker gaining code execution on a Block laptop. They aimed to test if prompt injection could operationalize a full attack chain. The operation focused on stealth and evasion to avoid detection by Block’s response team.
The Calendar MCP Vector and Invisible Prompt Injection Vector
The red team surveyed Block’s internal MCP marketplace. They found the Google Calendar MCP accepted external invites without email notification. Attackers could send invites with malicious event descriptions invisible to humans via zero-width unicode characters. Goose interpreted the text normally. The injection tricked Sonnet 3.5 into running a curl command piped into bash by framing it as an MCP update. The resulting payload downloaded an info stealer to an attacker server. Campaign 1 sent these invites but failed due to context window changes and model non-determinism.
Recipe Injection and the Successful Campaign
The team pivoted to Goose’s recipe feature, which allowed system prompt injection. Recipes base64-encoded content. The team embedded zero-width characters so the injection remained invisible when decoded. They updated the calendar invite with a link to a malicious recipe disguised as a tool feature. Users who clicked the link and ran the recipe executed the payload. The team then contacted the Goose development team via public channels with a fake RTL text bug report. The developers ran the recipe as part of their workflow, triggering the curl-bash command and deploying the info stealer.
Detections and Mitigations
The blue team caught the info stealer exfiltration through existing detections through existing controls. The red team implemented mitigations: stripping non-standard unicode characters from Goose input and recipes; displaying recipe content to users before execution, including hidden text; building a bad bash command detector using a bi-gram classifier with LLM evaluation; and adding command allowlisting certain commands. For Google Calendar they changed the policy so external invites require an accompanying email. This enables traditional email scanning for prompt injections and unicode characters.
Q&A
How do you handle false positives with prompt injection detectors? The team built a custom classifier that produces better results than open source ones, but noise remains a problem. LLM-based evaluation after classification helps. ▶ 27:54
What signs should a detection team look for in agent logs? Zero-width unicode characters showed up as weird characters in logs. Hunting for those characters became a detection method, but prompt injection detection overall has no one-size-fits-all solution. ▶ 28:51
Why did you target the Google Calendar MCP over other vectors? Calendar invites receive less scrutiny than email. The team looked for the lowest hanging fruit and found this vector worked without needing to pivot. ▶ 30:20
Notable Quotes
We needed something that was stealthy and that wouldn’t get caught. Josiah Peedikayil · ▶ 05:43
Goose was actually fixing our typo. Josiah Peedikayil · ▶ 21:42
the blue team ended up reaching out via this like red team deconliction workflow Josiah Peedikayil · ▶ 24:29
Key Takeaways
- Invisible unicode characters bypass human review but are interpreted by AI agents.
- Prompt injection testing must use realistic context windows and multiple.
- System prompt injections through recipes give attackers more reliable.