AI
Triage at Scale: Humans + Automation
Ryan Nolette of AWS said humans do not scale. His team added technology to reduce effort on triage. They made TLP rating a required field on every report to remove confusion about sensitivity. Gabriel Nitu at Splunk added a standardized submission template (summary, impact, proof of verification) to cut triage time. Jay Dancer at Shopify described the challenge of balancing report volume with relationship quality: transparency and quick communication are critical, but volume sometimes overwhelms the team.
The AI Slop Problem
Ryan Nolette reported a flood of AI-generated reports that sound convincing but contain no real vulnerability. One report was a 400-word essay with no proof. Jay Dancer said researchers now run AI tools against programs, producing submissions that are indistinguishable from human-written ones when the finding is real. Gabriel Nitu warned that AI slop forces triagers to spend hours filtering before they reach high-quality reports. Tyson Laa Deng added that some researchers submit theoretical ideas without impact. The panel’s consensus: use AI to learn, not to report.
Non-Monetary Rewards
AWS runs its entire bug bounty without public monetary awards. Ryan Nolette listed swag: lightsabers, challenge coins, custom badges for valid findings, account credits, and exam vouchers. Gabriel Nitu said Splunk invites out-of-scope submitters to a private program and awards a bonus for the original informative report. Jill Moné-Corallo shared that Shopify is building an additional reward tier on top of cash bounties, responding to researcher requests for more than just money.
Evolving Researcher Relationships
Tyson Laa Deng said PayPal started with email‑based submissions. Now his team maintains a private chat with top hackers and personally checks in if a regular submitter goes silent. Gabriel Nitu pays a triage bounty right after validation, cutting the wait for researchers. Ryan Nolette sits in Slack and Discord channels to answer questions in real time. Jill Moné-Corallo spent months building an external bug bounty site with clear policy and transparent communication. All panelists stressed that if the relationship hasn’t changed in eight years, it’s wrong.
Future of Bug Bounties
Ryan Nolette predicted that collaboration between researchers and programs will remain the foundation. AI won’t replace the human relationship. Gabriel Nitu sees a trend toward responsible disclosure: researchers increasingly coordinate with vendors before public release to protect customers. Tyson Laa Deng expects industry‑focused hacking clusters — researchers who deeply understand fintech, healthcare, or e‑commerce. Jill Moné-Corallo warned that bug bounty is a marathon, not a sprint; quick money seekers wash out.
Q&A
How do programs handle AI‑generated reports and automated attack tools like expo? Gabriel Nitu responds with a warning: stop sending AI slop or risk being banned from the program. ▶ 53:03
What happens when a researcher argues a report marked “informative” or “working as expected”? Ryan Nolette relies on publicly documented compensating controls and communicates transparently. Jay Dancer described using a three‑strike rule before mediation. ▶ 55:33
Notable Quotes
bug bounty works but only as a supplement to a strong internal security foundation. Otherwise, you’re just paying for the illusion of security. Gabriel Nitu · ▶ 50:32
data without context is not actionable intelligence. Ryan Nolette · ▶ 49:44
if you’re in it only for the money that’s not going to work for you. It’s a marathon. It’s not a sprint. Jill Moné-Corallo · ▶ 51:16
If your relationship is the same now as it was 8 years ago, it’s wrong. You have to constantly evolve. Ryan Nolette · ▶ 44:14
bounties is not easy, is not an easy fast cash game, it’s more like gold mining. Gabriel Nitu (paraphrasing a LinkedIn post) · ▶ 16:36
Key Takeaways
- Triage scales by standardizing report fields and using automation for repetitive questions.
- AI‑generated reports flood programs; programs respond with warnings and bans.
- Non‑monetary rewards like challenge coins and custom badges complement cash bounties.
- Direct, transparent communication with researchers builds trust and reduces friction.
- The future favors industry‑focused hackers who understand regulatory and business context.
About the Speaker(s)
Gabriel Nitu is a Splunk Offensive Security Engineer with over 9 years of experience in vulnerability research, incident response, and scaling bug bounty programs.
Jill “thejillboss” Moné-Corallo is the Bug Bounty Leader at Shopify. She previously led Bug Bounty and Product Security Incident Response at GitHub and was a Senior Product Security Engineer at Apple. She holds a B.S. in Cybersecurity from Mercy University and founded Glass Firewalls, a conference for women in bug bounty.