AI
Where CodeQL’s Built-In Rules Miss Bugs
CodeQL has found 418 bugs and counting, but testing it against recent high-risk vulnerabilities exposed a consistent failure mode: false negatives. Yuan Luo traced those misses to two root causes. First, the built-in source and sink models don’t cover every API in every framework, so tainted data that flows through an unlisted function passes undetected. Second, certain Java language features, including cross-thread calls, reflection, and pass-by-value semantics, break the data flow graph before it reaches a sink.
“so far it has found 418 bugs” – Yuan Luo
Automating Source and Sink Discovery with LLMs
Source and sink functions are API methods in third-party frameworks, and their implementations live in open-source repositories. That means they can be scanned automatically. Luo’s team built a three-agent pipeline: a discover agent scans framework source files at the file level, prompting the model with characteristics tied to the bug type (for SSRF, the sink sends HTTP requests). A judge agent then applies expert rules, like checking that the function is public and returns non-boolean propagated data. A validation agent runs the new rules against the top starred dependent projects to confirm the sink is actually used in the wild.
“both methods are pretty labor intensive” – Yuan Luo
Patching Cross-Thread Data Flow
When taint passes through a Java Runnable, CodeQL’s default analysis loses the thread. The flow breaks at the constructor call. Luo’s fix uses CodeQL’s jump step to connect the constructor call directly to the this parameter of the run method. But Runnable instances aren’t always initialized through the constructor. When taint arrives via an assignment before start() is called, the fix jumps from the start() caller to the run method’s instance parameter. When taint arrives after start(), the fix identifies a post-update node with a store operation and jumps from there.
Handling Reflection and Pass-by-Value
Java reflection breaks CodeQL in two places. The engine can’t resolve which method invoke actually calls, so Luo’s team added two resolution strategies: tracking the method instance through global data flow, and matching on parameter count and types. additionalValueStep also can’t call the data flow interface without triggering non-monotonic recursion, so their fix duplicates the data flow implementation and wires reflection analysis to that copy.
Pass-by-value creates aliasing gaps when a tainted object is stored across multiple copies. The fix traces store operations using global data flow, identifies post-update and non-post-update nodes, then connects them with jump-step edges.
Results: More Flows, More CVEs
The pipeline found 190 source and sink functions across 18 Go frameworks. Scanning over 5,000 projects with the new rules raised detected data flows by more than 15%.
“we have found about 190 source and sync functions across 18 Go frameworks” – Yuan Luo
The Apache Traffic Control SQL injection CVE walkthrough (21:58) shows the method in practice. CodeQL had the project enabled but missed a SQL injection because queryRowx, the actual sink, wasn’t in its model. Scanning the sqlx framework surfaced it. One added source function unlocked detection of three separate CVEs. The reflection fix reproduced another previously undetectable CVE.
Notable Quotes
so far it has found 418 bugs Yuan Luo · ▶ 2:00
both methods are pretty labor intensive Yuan Luo · ▶ 3:24
we have found about 190 source and sync functions across 18 Go frameworks Yuan Luo · ▶ 21:40
Key Takeaways
- Semantic analysis of source code is well-suited for LLM assistance, and combining both is a productive research direction.
- CodeQL’s data flow engine is a strong learning reference but has real gaps in cross-thread, reflection, and pass-by-value handling.
- Adding 190 new sources/sinks across 18 Go frameworks raised detected data flows by over 15% across 5,000 scanned projects.