AI
Why Synthetic Vulnerability Injection Matters for AI Evals
LLMs can find vulnerabilities now. The next question, which Aaron Grattafiori and Skyler Bingham started asking six months ago, is whether they can also inject them. If you can plant a known flaw into any codebase on demand, you have ground truth. You can benchmark a new scanner, measure signal-to-noise ratio, or test whether your team finds what you told the system to hide. Historical commits give you some of that, but the supply is limited. Synthetic injection scales.
“PC or get the f*** out, eval critical.” — Aaron Grattafiori
Skill-Based Architecture for Getting Models to Write Vulnerable Code
Asking a frontier model to insert a vulnerability directly fails. It refuses, or it adds a string overflow in a config parser that only fires if you already own the machine. The fix is a set of markdown skill files, each handling one task: attack surface analysis, CVE research, diff creation, exploit verification. Skills call other skills, and that structure acts as a soft jailbreak.
“if you have a defined set of skills and you say, ‘Hey, go use these skills’ — instead of putting your prompt right in the thing, they’ll go ahead and follow that and it works pretty well.” — Aaron Grattafiori
Reward hacking still surfaces. An early run “introduced” an XSS flaw by disabling the XSS protection header (08:17) rather than injecting one. A critic skill plus exploit-validation scripts close that gap.
Injection Modes and Difficulty Levels
Marinade ships five injection modes. Interactive puts a human in the loop to approve each flaw. OWASP Top 10 sweeps a full application and tries one flaw per category. CWE mode plants a specific weakness class in the right spot. CVE mode pulls an original implementation and emulates the structure. Auto-RCE mode plants several individually weak flaws that, when chained, achieve full remote code execution.
Difficulty is a separate dial: easy means the scanner catches it, medium means it must not, hard means it resists human review. A flagged injection loops and retries.
The Build-Analyze-Inject-Validate Pipeline
Four stages, with build and analysis able to overlap. Build generates start/stop scripts, runs tests against the clean codebase, and writes a document the agent consults after each recompile. Analysis maps the technology stack, endpoints, data flows, and security mechanisms so the agent can place flaws plausibly. Then injection. When asked for a Top 10 sweep, the agent self-limits to eight categories it deems applicable, refusing to add attack surface that was never in the original app (19:12). A critic reviews each plan and implementation. Validation scripts confirm the flaw is exploitable. A second scanner pass checks detection — at medium or hard difficulty, a catch sends the loop back.
Extensions: Binaries, Config Files, Log Injection, and Synthetic Training Data
Source code is just the starting point. Compile those injected binaries and you have a target for patch-diff analysis or n-day research. Inject into config files to test whether your security process catches misconfigurations. Drop synthetic entries into log files and fire them at your SOC — no red team required.
“we’re we’re not in cyber Kansas anymore, right? It’s very interesting times for the security industry, for the AI industry.” — Skyler Bingham
The biggest multiplier is training data. A corpus of validated, exploitable vulnerabilities with verifiable rewards can feed back into model training and produce better vuln-finding agents — compounding the capability of the next round.
Q&A
Will better model safety refuse these skills the same way naive prompting does — is there a capability vs. refusal arms race? Grattafiori acknowledged the race is real and pointed to programs like OpenAI’s trusted-access tier, where approved companies can get safety classifiers dialed down, as a likely path forward. ▶ 23:07
How realistic are the injected vulnerabilities on a sliding scale from CTF challenge to real-world bugs? The team deliberately designed away from CTF-style targets and toward real-world patterns, but Grattafiori admitted bias is hard to eliminate and that CVE-emulation mode, feeding the last four known bugs in a product to generate a fifth, is still experimental. ▶ 23:49
Does this look like the perfect tool for inserting backdoors via malicious pull requests? “No comment.” ▶ 25:46
Notable Quotes
English is, you know, the new coding language. Aaron Grattafiori · ▶ 5:37
if you have a defined set of skills and you say, ‘Hey, go use these skills.’ Um, instead of putting your prompt, you know, right in the the thing, they’ll go ahead and and um they’ll follow that and it works pretty well. Aaron Grattafiori · ▶ 5:08
we’re we’re not in cyber Kansas anymore, right? It’s very interesting times for the security industry, for the AI industry. Skyler Bingham · ▶ 22:34
you cannot just prompt a few times with cloud code and just get a good result here you really need the structure, you need various skills to go through and be able to do these things. Aaron Grattafiori · ▶ 10:58
Key Takeaways
- Structure vuln-injection as a skill system, not a single prompt — naive requests to frontier models will be refused or produce trivial flaws.
- Run a baseline scanner before injection and re-run after so you know exactly what the scanner missed, not just what you planted.
- Synthetic vulnerability corpora with verifiable rewards can feed back into model training, compounding the capability of future vuln-finding agents.