AI
When High-Fidelity Detections Don’t Scale
Nicole Grinstead and Swathi Joshi joined Netflix in fall 2017 with a vision: build high-fidelity, true-positive detections that eliminate the need for a SOC. Netflix’s homogeneous environment and in-house detection platform gave them control over detection types and telemetry. The alert queue grew anyway. Employees, attack surface, and compliance obligations expanded faster than automation could keep up. The sockless future lasted only briefly before the team rebuilt a SOC.
The Tier-One Pipeline Must Survive
Dean De Beer argued every CISO has wanted to replace tier-one analysts for 20 years. What they actually need is tier-one analysts functioning at tier-two or three levels. Eliminating entry-level roles kills the pipeline for senior analysts. Within three to five years, organizations would have only validation staff checking whether AI agents function, not whether they function correctly. Tier-one roles shift from pattern recognition to ontology engineering, data structuring, and trust authorization for agentic systems.
Token Cost vs. Human Cost: Measuring What Matters
Bryan Fite warned that AI adoption shows two extremes: blind experimentation without lifecycle cost analysis, and fear-driven prohibition that stifles innovation. He recommended targeting middle-ground pain points like 20-click workflows that can shrink to two clicks. Swathi Joshi added that false-positive reduction, alert deduplication, and parallel escalation throughput are concrete productivity metrics. The goal is reducing cognitive load on analysts while expanding coverage, not closing alerts faster.
Structured Data and Business Context Before Agents
Dean De Beer emphasized that SOCs must bring in data from security and operational sources, GitHub, AWS, SharePoint, HR data, to give investigations organizational context. Language models lack this context by default. A critical incident in one organization is a policy violation in another. That nuance must be codified and fed to the system upfront. Data management and business context engineering become the primary human-in-the-loop responsibilities before any agent can operate autonomously.
Narrow Models and Guardrails Beat General-Purpose LLMs
Bryan Fite recommended small, focused language models over large ones for SOC tasks. A model trained exclusively on IP and network data cannot generate harmful or irrelevant outputs. Swathi Joshi reported that training a model on localized business context, payment workflows, anomaly definitions, took her team 10 to 15 months before achieving human-assisted and fully AI-driven decisions. Structured telemetry produces reliable reasoning. API-first architectures and streaming pipelines are replacing batch-oriented SOC tooling.
Q&A
What happens after the first major breach blamed on an AI SOC triage miss? Dean De Beer said analysts already miss alerts under heavy workloads, and AI tools won’t provide 100% accuracy. Checks and balances remain essential. ▶ 36:12
How should SOC teams influence other orgs to enable AI-powered operations? Swathi Joshi suggested reducing dependency on other teams by automating their SOPs, turning their problems into automated workflows. ▶ 41:46
Notable Quotes
I’d say we operated sockless for a while Swathi Joshi · ▶ 2:23
I think for the past 20 years uh every cso or CISO at every organization has wanted to replace tier one Dean De Beer · ▶ 4:36
if I can take 20 clicks to two clicks that seems like it would be pretty easy Bryan Fite · ▶ 14:59
all of those attacks that work on humans um actually work on these weird machines, too Bryan Fite · ▶ 40:36
we were joining Netflix and we had this vision we are going to be totally sockless all our detections are going to be high fidelity true positives Nicole Grinstead · ▶ 1:56
Key Takeaways
- Netflix’s sockless vision failed because alert volume outgrew automation; human SOC roles evolve but do not disappear.
- Measuring AI success requires concrete metrics like false-positive reduction and click reduction, not alert closure rates.
- Structured data and business context must precede agent deployment; small focused models reduce hallucination risk.
About the Speaker(s)
Nicole Grinstead is a seasoned cybersecurity leader currently serving as the Senior Director of Product, Enterprise and Application Security at Roblox. In this role, she oversees the proactive security of Roblox’s global platform, managing everything from core infrastructure hardening to broader security initiatives.
Swathi Joshi currently leads the SaaS Information Security team at Oracle which has a charter to secure SaaS Applications. Before that she led Netflix’s Detection and Response team which focuses on managing the inevitable security incidents that arise and building detection pipelines for the streaming platform.