AI
The Vuln Apocalypse: Scale Forces a New Strategy
“we are not far off from a world where there will be an open-source full hacking tool where you can type in go hack Google and a week later it’ll come back with a root prompt.” — Heather Adkins
About $1 billion in VC money is funding vulnerability discovery, pen testing, and red team simulation startups. NVD already carries a 30,000-entry unanalyzed backlog. CVEs grew 35% between 2024 and 2025 — and not every discovered bug gets a CVE. Agentic frameworks will soon find every vulnerability in every system. CVSS scoring will stop being meaningful. The numbers point one direction.
“we simply must eliminate every software vulnerability on Earth.” — Heather Adkins
Big Sleep: Recreating Elite Researcher Expertise at Machine Speed
Pointing an LLM at code and asking it to find bugs works. Even older models do it reasonably well. Big Sleep sets a higher bar: find the bugs that take an elite researcher roughly a month to locate and another 22 days to exploit, according to a 2017 RAND study. The goal is not shallow findings. It is the deep, embedded class of vulnerability that only the top researchers reach.
“we do not want to produce AI slop with the bugs we find.” — Heather Adkins
To get there, the team modeled how Project Zero researchers actually work: deep familiarity with a specific codebase, study of past vulnerabilities in that code, and variant analysis. That context becomes phase one of Big Sleep. Without it, the system is just prompting. With it, the system reasons about code the way an expert does.
Big Sleep Architecture: Context, Agentic Loop, and Zero False Positives
Big Sleep runs in three phases. Phase one loads the codebase into context — past vulnerabilities, variant history, architecture — the same knowledge a Project Zero researcher builds over months. Phase two is an agentic loop: the model forms a hypothesis, writes test scripts using a debugger and Python interpreter, runs them, reads the results, and revises. Once a crash looks real, phase three builds an exploit. That proof of vulnerability is what drives the false positive rate to zero. Every report that ships includes working exploit code and a step-by-step explanation any developer can follow. See the Big Sleep zero-false-positive results (08:55) — end-to-end, fully automated, no human in the loop, and all but five of the reported bugs were fixed as of the talk date.
Code Mender: Automated Patching with Multi-Layer Verification
Code Mender takes a verified Big Sleep finding and turns it into a merged patch. Every candidate fix must clear four verifiers: fuzzing before and after the patch confirms functionality survives, formal verification proves the patched section is still functionally equivalent, a second fuzz round checks the original vulnerability is gone, and an LLM reviews the diff against a crafted pre-prompt. Candidates that fail feed their failure context back to the agent, which generates a revised set. Flynn walked through the Code Mender 178 open-source fixes (14:45) — patches merged into libwebp and Chrome pointer hardening. “We believe developer trust is what drives adoption,” Flynn said. Nothing ships until it passes.
Q&A
How similar or different is this from OpenAI’s adwork that was recently released? No head-to-head comparison yet — neither team has published full details, though both use agentic reasoning under the hood. ▶ 17:00
How many of these techniques apply to vulnerability classes with less clear signal than a segfault — things harder to verify than memory bugs? The techniques carry over to web and shallow vulnerabilities; the verification methods differ, but the team is actively building support for input-validation and web vulnerability classes. ▶ 17:55
How are you thinking about business logic vulnerabilities that don’t have a clear signature but depend on application context? Current research targets infrastructure components like V8 and ffmpeg rather than business applications, and how the methodology translates to business logic is still unknown. ▶ 19:07
Notable Quotes
we simply must eliminate every software vulnerability on Earth. Heather Adkins · ▶ 3:00
we are not far off from a world where there will be an open- source full hacking tool where you can type in go hack Google and a week later it’ll come back with a root prompt. Heather Adkins · ▶ 0:59
we do not want to produce AI slop with the bugs we find. Heather Adkins · ▶ 5:38
big sleep is finding things the fuzzers are missing. Heather Adkins · ▶ 18:56
Key Takeaways
- Build exploit proofs, not bug reports — zero false positives require working exploit code before a finding ships.
- Treat patch quality as a trust problem: developer adoption collapses if automated fixes introduce regressions or alien idioms.
- Pair every discovery engine with a fix engine now, before the vuln apocalypse outpaces human patch capacity.