AI
Cross-Service Prompt Injection Correlation: The Problem
Microsoft runs Azure AI Foundry, GitHub Copilot, and M365 Copilot as separate products with separate safety stacks. No threat signal crosses between them. When an attacker finds one effective prompt injection and sprays variants across all three, each service evaluates the attack in isolation. One detection doesn’t protect the others.
“We need a cross-service boundary threat correlation that respects user privacy because we cannot just share the raw prompt from one service to another service.” — Waris Gill
Binary Shield Architecture: Four-Step Fingerprinting Pipeline
Each prompt that enters Binary Shield passes through four steps before any fingerprint leaves the service. First, PII redaction strips names, social security numbers, and email addresses, replacing them with placeholders. Second, the cleaned text feeds into an embedding model (OpenAI’s text-embedding-3-large in the Microsoft implementation) to produce a high-dimensional float vector. Third, quantization compresses that vector to zeros and ones, cutting memory overhead and making the pipeline deliberately one-way. Fourth, differential privacy flips bits at random, controlled by a single parameter: epsilon.
“When epsilon is large the bits don’t change. So that means that there’s very little privacy but high utility. And then the inverse is also true. When epsilon is small the bits are more random. So there’s a lot of privacy but very little utility.” — Natalie Isak
Accuracy vs. Privacy Budget Trade-Off and Speed Gains
At epsilon 0.5, detection accuracy drops to 0%. Flip enough bits and the fingerprint is noise. As epsilon grows, noise decreases and accuracy climbs toward dense embedding performance.
“When epsilon is large the bits don’t change. So that means that there’s very little privacy but high utility. And then the inverse is also true. When epsilon is small the bits are more random. So there’s a lot of privacy but very little utility.” — Natalie Isak
The binary format also cuts search time. Against a growing corpus, dense floating-point embeddings show significant overhead. Binary Shield’s quantized fingerprints run 36x faster on the same threat correlation task.
Live Demo: Building and Deploying Fingerprints Across Services
Isak walks through each pipeline step in code: strip PII (names, SSNs, emails) using the open-source Presidio library, embed with text-embedding-3-large, quantize floats to zeros and ones, then inject noise via epsilon. She sets epsilon per organizational context, not by formula. The Hamming distance matrix (10:48) makes the payoff concrete: three variants of “ignore all system instructions” cluster with low distances from each other, while a benign weather query sits far outside that cluster. Then she runs all four prompts through three simulated services with different blocklists. Only service alpha catches the attack. After alpha fingerprints it and broadcasts, beta and gamma immediately gain detection coverage (14:24) with no changes to their safety stacks.
Q&A
Are the components going to be open sourced? The paper is already public and the notebook was quick to build with Claude Code, so open sourcing the components is something they will look into. ▶ 15:26
When you quantize embeddings to zeros and ones you lose a lot of semantics — how do you address that? And how do you refresh the privacy budget as fingerprints consume it? High-dimensional embeddings (768+ dimensions) retain enough semantic signal in binary form to catch attacks reliably, and there is no prescribed budget refresh — each organization works with its own privacy and legal teams to pick an epsilon that balances noise and utility. ▶ 15:48
Could you generate variants of a caught malicious prompt using a large model and pre-populate the registry to get ahead of attacks instead of catching them one by one? Synthetic variant generation is worth doing, but the fuzzy fingerprint already catches small perturbations around the original prompt, so the system is not whack-a-mole — one fingerprint covers a ring of related variants, and the threshold can be widened if broader coverage is needed. ▶ 17:35
Notable Quotes
we need a cross service cross service uh boundary thread correlation uh that respects user privacy because we cannot just share the raw prompt from one service to another service Waris Gill · ▶ 1:05
when epsilon is large the bits don’t change. So that means that there’s very little privacy but high utility. And then the inverse is also true. When epsilon is small the bits are more random. So there’s a lot of privacy but very little utility. Natalie Isak · ▶ 9:20
we’re not catching um verbatim prompts one by one. We’re capturing the prompt you want to catch and then small perturbations as well. Natalie Isak · ▶ 18:34
Key Takeaways
- Generate privacy-preserving binary fingerprints from prompts so threat intelligence can cross service boundaries without exposing raw user data.
- Set epsilon in collaboration with legal and privacy teams — no universal value exists, and the right budget depends on regulatory context and data sensitivity.
- One detected attack fingerprinted and broadcast to all services closes coverage gaps across divergent safety stacks without requiring a full stack rewrite.