AI
Naive LLM Prompting and Why It Fails for Vuln Discovery
When LLMs first dropped, security researchers did the obvious thing: pasted code in and asked for vulnerabilities. It worked. Then it didn’t. Behrens and Cassel ran that experiment starting in February 2025 and hit the same wall everyone else did.
“consistency was difficult, variance was huge, false positives kept coming up” — Justice Cassel
One prompt change made the model brilliant. The next made it skip entire vulnerability classes. Sessions crashed mid-scan and lost all findings. The tooling felt magic until it didn’t, and chasing “one more prompt” burned time without producing reliable results.
Building a Benchmarked Evaluation Framework
Before running any agent configuration, Behrens and Cassel built a benchmark: a fixed codebase with 41 known true positives, seeded dead code, false-positive fixtures, and commonly misinterpreted vulnerability paths. Without that ground truth, any result is just noise. They also wired in automated metrics collection so they could re-run tests cheaply after every change. The need was real: one prompt line change could flip a model from sharp to useless, and a single added tool that worked on the first run might fail on the second. Continuous measurement was the only way to tell the difference between a real improvement and a lucky run.
Agent Topology Experiments: What Actually Moves the Needle
Four experiments with the same 41-vulnerability benchmark separated what actually works from what sounds reasonable. A monolithic super-agent with a large security rule file found 36 of 41 true positives at the lowest cost. Splitting into specialized agents found more, but cost more.
“a monolithic agent is cost-effective, but a multi-agent configuration finds more true positives” — Scott Behrens
Dedicated false-positive triage made the biggest single difference: removing it dropped correct severity assignments from 74% to 26%. Grouping agents by vulnerability category (injection, network, etc.) looked efficient but missed 7 of 41 findings — they suspect the model overfits on category names. Adding source-to-sink tracing with enumeration and discovery gates cut costs by roughly 26% while improving depth, because agents only received files and sinks relevant to their job.
Real Disclosures and Operational Failures
The failures were mundane: a half-day experiment scrapped because the API didn’t support custom tools, a 45-minute scan killed by auto-compact, a session that silently turned a config file into JavaScript. Each one exposed a gap the architecture had to close.
Against open-source projects with over 5,000 stars, the team found a TOCTOU double-spend in a trading platform and a command injection in a VOIP stack — RCE over fax. Opus 4.6 dropped mid-project and the numbers reframed everything: solo, it found 40 of 191 true positives on an XL codebase (13:16). An orchestrated workflow using a less powerful model found 116.
“for large code bases being really thoughtful about your context management and your workflow can result in higher fidelity results even more than a really powerful model like Opus” — Scott Behrens
The Source-to-Sink Orchestration Architecture
The production workflow starts with deterministic enumeration: walk the file tree, strip test artifacts and build outputs, then run AST, LSP, Semgrep, and CodeQL to map sources and sinks before any model touches the code. Discovery gates decide which specialist agents activate — no database means no SQL injection agent runs. Each agent gets only the files relevant to its sink class, batched by cross-imports so analysis stays local and token-cheap. Downstream, programmatic schema enforcement catches hallucinated severity ratings or nonsense vulnerability classes and forces the agent to re-evaluate against a strict enum. A dedicated false-positive triage agent runs last, discarding anything that doesn’t hold up.
Live Demo: Flask App Scan End-to-End
The demo targets a simple Flask app with a handful of known vuln classes. Phase one filters out tests, .git, and large images before anything reaches a model. Flask app demo — file enumeration and parallel discovery phases (16:43) Phase two runs static analysis and data-flow tracing in parallel, dismissing unreachable candidates in real time, then flags two data flows worth following: a request parameter rendered into HTML output and a database read rendered into HTML output.
Once the XSS agent reports two findings (reflected and stored) and the SQLI agent reports two more, the false-positive agent steps in. Demo — FP agent dismisses allow-list-validated SQLI finding (19:05) It catches that the event_type parameter is validated against an allow list before touching the SQL query, marks that finding non-exploitable, and leaves one real SQLI to triage.
Q&A
Did you play around with agent sequencing or repetition to see if results varied? They have tried chaining agents of varying cost (Haiku to Sonnet to Opus) based on triggers, which produces better results but also significantly higher cost, and the right trade-off depends on how much you’re willing to spend per repo scan. ▶ 21:17
Security prompting and getting the right results is hard — any tips or tricks? Give the model ground rules and explicit constraints on what not to do, but leave room for it to discover variants — and for rare vuln classes like TOCTOU, include the class name directly in the prompt because LLMs are trained on data volume and will miss low-coverage classes without explicit priming. ▶ 23:03
Do you have a separate surface for PRs, or do scans build on each other? They have an MCP that triggers on pull requests and an incremental diff feature that scans only the changed code, with plans to feed saved traces and false-positive decisions into PR reviews so the model can decide whether to do additional exploration without re-scanning the full codebase. ▶ 24:35
Notable Quotes
consistency was difficult, variance was huge, false positives kept coming up Justice Cassel · ▶ 1:00
a monolithic agent is cost-effective, but a multi-agent configuration finds more true positives Scott Behrens · ▶ 7:13
for large code bases being really thoughtful about your context management and your workflow can result in higher fidelity results even more than a really powerful model like Opus Scott Behrens · ▶ 13:46
very rarely have we actually seen Opus find meaningful time of check, time of use without having time of check, time of use in its prompt Scott Behrens · ▶ 24:03
if you’re only looking for OWASP top 10 with a vanilla model, you’ll find it. But the minute you start stepping into more esoteric vulnerabilities, providing that context in a vanilla prompt or in a workflow results in significantly better coverage Scott Behrens · ▶ 24:14
Key Takeaways
- Run a dedicated false-positive agent post-scan — it raised correct severity assignments from 26% to 74% in their benchmark.
- Workflow architecture and context management beats raw model power: an orchestrated workflow found 116 TPs where Opus solo found only 40 on the same large codebase.
- Explicitly include rare vuln classes (TOCTOU, logic flaws) in your prompts — LLMs are trained on data volume and will miss low-coverage classes without explicit priming.