AI
Start with the Problem, Not the Vendor
Sharma opened with the “security tools tragedy”: a flashy demo, a pilot approved, then within two weeks engineers complain about false positives, the security team drowns in configuration, and leadership asks why a six‑digit contract was signed. He traced the root cause to shopping without a list. Before talking to a single vendor, security engineers must define the exact business outcome and align it to a gap. For example, a company expanding into Europe needs GDPR controls. Once the gap is clear, you decide the kind of tool needed.
Build a Pitch Like a Song: Hook, Chorus, Bridge, Finale
Sharma structured his vendor pitch as a four‑part song. The hook is a single sentence about the problem, never the solution. For the supply chain tool, the hook was “We are resolving 13 critical vulnerabilities per year out of thousands, and our mean time to remediate is close to a calendar year.” The chorus backs the hook with baseline numbers — open alerts trending up and to the right. The bridge connects the problem to the tool’s outcome, not its features. Nobody cares about reachability analysis; they care about cutting 200 CVEs to five exploitable ones. The finale is the success criteria, a pseudo‑contract that holds the team accountable before any dollar is spent.
Handle Tough Questions and Pilot with Opt-In Teams
Sharma said pushback is a good sign; silence means the audience decided “no” already. The three questions he always gets: Will this break production? What is the burden on engineering? How is it different from what we already have? He answers by showing the experiment design that will prove safety. For the pilot, he recommended a 30‑60 day timebox, an opt‑in model with two or three teams of different tech stacks, and success criteria written before day one. Mandatory pilots produce bare‑minimum effort and dishonest feedback.
Measure Technical Metrics and Developer Adoption
Sharma separated measurement into two buckets: technical and adoption. Technical metrics include false‑positive rate, detection time, and remediation time. Adoption metrics — often forgotten — covered whether developers look at alerts, act on them, turn them off, or ask for help. In the supply chain pilot, one commit was alerted out of many, showing low noise. Developers used a PR‑level bot and a browser extension to see dependency risk before writing code. Sharma counted Slack complaints as real signals too.
Roll Out Team by Team and Keep Monitoring
After a successful pilot, Sharma warned against a big‑bang rollout. He advised going team by team: start with the pilot teams, then teams with similar tech stacks, then the rest, each phase two weeks with a tight feedback loop. Turn early adopters into salespeople — team A convincing team B carries more weight than security evangelizing. After deployment, tools need continuous monitoring, policy tuning, and a feedback loop with the vendor. Every 12 to 18 months, run a mini‑version of the same evaluation because tools get old fast.
Q&A
Does the same process apply for big and small purchases? Yes, the trigger may change but the data‑driven process stays the same. ▶ 36:47
How many pilots per year? Around four to five, including both small and big tools. â–¶ 37:11
How do you get leadership buy‑in to protect the pilot from shifting priorities? Align on the problem with your own leadership first, before looking at any product. Then expand alignment to other stakeholders. ▶ 37:31
How do you decide build versus buy? If a custom solution is easily achievable, the needle tips toward build, especially with AI. Otherwise evaluate team time and benefits. â–¶ 38:05
Should you write failure criteria as well? Yes. Define explicit conditions that stop the pilot immediately. â–¶ 38:58
Notable Quotes
13 critical vulnerabilities in a calendar year out of thousands Saurabh Sharma · ▶ 8:33
CVSS scores … like horoscopes for vulnerabilities Saurabh Sharma · ▶ 11:52
developers started ignoring the alerts … once they start ignoring your tool, it’s incredibly hard to get that trust back Saurabh Sharma · ▶ 23:04
the failure would be to actually go ahead and like waste another another year on the tool which you have already determined which it does not work Saurabh Sharma · ▶ 29:33
I make the early adopters the sales people Saurabh Sharma · ▶ 33:36
Key Takeaways
- Align tool selection to business outcomes, not vendor features.
- Pitch using data: hook, chorus, bridge, and success criteria.
- Run 30‑60 day opt‑in pilots with pre‑defined success and failure criteria.
- Measure both technical metrics and adoption metrics, including qualitative feedback.
- Roll out team by team and continuously monitor and tune policies.