The AI Productivity Gap and What AI-Native Actually Means

▶ Watch (02:15)

Thousands of CEOs surveyed two weeks before this talk reported that AI had no measurable impact on productivity. Guido’s explanation: they gave people tools without changing the system. Everyone got a ChatGPT license, leadership waited for the numbers to move, and they didn’t.

“That’s the gap between AI-assisted and AI-native. One is a tool, the other is an operating system.” — Dan Guido

AI-assisted is level one: draft emails, generate boilerplate, same workflows, slightly faster. AI-augmented is level two: agents enter the loop, AI does the first code review pass, the human does the second. AI-native is the structural shift — the organization is designed from the start assuming AI is a co-participant. At Trail of Bits, that means 14 years of audit knowledge encoded into agents every engineer can invoke.

Four Psychological Barriers to AI Adoption Inside Expert Organizations

▶ Watch (05:30)

Guido launched the initiative with 95% of Trail of Bits against him. Four biases drove it. Self-enhancing bias: experts credit intuition for success, so a model that outperforms them reads as an attack. Paul Meehl showed a crude linear model built from an expert’s stated variables beats that expert. Consistency wins. Identity threat follows: “I spent 20 years doing this” collides with automation. Cooks in a Stanford study rejected the same device when framed as doing the cooking for them. Opacity is third: engineers trust human judgment they can’t inspect over AI output. Fourth: one bad result and people quit.

The Change Management System: Matrix, Hackathons, and Leadership Visibility

▶ Watch (08:56)

The maturity matrix addressed self-enhancing bias directly. Nobody can argue they’re good enough when a visible ladder shows where they actually stand, and when peers are climbing it, the passive 50% starts moving. For identity threat, Guido reframed what it means to be an expert. A senior consultant who writes a constant-time analysis skill isn’t being replaced. That knowledge becomes permanent, reusable by the whole team. Level three on the matrix isn’t who uses AI the most — it’s who builds new tools.

“The identity shifts from I don’t need AI to I’m the one who makes AI dangerous.” — Dan Guido

Hackathons replaced mandates. Two-to-three day sprints with one objective, no forced adoption. And Guido made his own use visible every day, because the passive majority watches what leadership does, not what it says.

The Five-Part Technical System: Standardization to MCP-Connected Agents

▶ Watch (12:00)

Five parts, all required. Trail of Bits standardized on Claude Code, wrote an AI handbook specifying which tools are approved on client code, and built public skills repos to capture hackathon output. A curated marketplace filters third-party plugins before they reach engineers.

“If you want adoption, you need a safe supply chain. Basic enterprise thinking.” — Dan Guido

Sandboxing is a menu: dev containers, single-use Digital Ocean VMs, or native macOS isolation. No package under 7 days old installs anywhere in the company, enforced via MDM (20:10). Once guardrails are in place, internal tools connect via MCP. Slither, the Ethereum static analyzer, runs as an MCP server (20:46). Agents call it reliably. The org governs how.

Results: 200 Bugs per Week and Compounding Expertise at Scale

▶ Watch (21:05)

After roughly one year, Trail of Bits has 94 plugins, 201 skills, and 84 specialized agents (21:11) encoding 14 years of audit knowledge. On qualifying client engagements, discovery rates went from 15 bugs a week to 200 (22:02) — one auditor running a fleet of agents in parallel, not a faster human. Across all engagements, 20% of reported bugs were first found by AI. Sales reps average $8 million in revenue, roughly double the consulting industry norm. The models improve monthly. The skills grow weekly. Guido’s read: “A year from now these numbers are going to look very different.”

Open Problems: Private Inference, Prompt Injection, and the Shifting Consulting Model

▶ Watch (23:20)

Three problems remain unsolved. Private inference: local models are improving, but the quality gap between Opus 4.6 and a local Haiku is still obvious. Guido is evaluating tinfoil.sh and similar confidential cloud inference providers rather than buying on-premise hardware. Prompt injection gets a blunt fix — deny web access for high-sensitivity clients and block external resources in internal tools. Longer term, Trail of Bits is testing agent-native shells like Agent.sh that enforce restrictions at the kernel syscall layer instead of a CLAUDE.md file. The third problem is the business model itself.

“In a future where everybody has an arsenal of agents and some people can outperform others by 200x, I think you have to start billing by expertise and by results.” — Dan Guido

Hourly billing may not survive that gap. Guido expects Trail of Bits to change how it prices services within 6 to 12 months, though the exact model is still open.

Q&A

Do employees also fear being replaced or mass joblessness — not just identity threat? Guido sidestepped the question, saying Trail of Bits is obelisk-shaped with only senior staff, so AI enhances rather than displaces them. ▶ 27:29

How do you guard against rug-pull / supply-chain attacks on your blessed MCPs and skills list? All external skills import through the curated allow-list repo, updates require a pull request review, and automated tooling lets any engineer submit a skill from a tweet for review and deployment in about 5 minutes. ▶ 28:23

Notable Quotes

That’s the gap between AI-assisted and AI-native. One is a tool, the other is an operating system. Dan Guido · ▶ 3:02

It’s not a tool, it’s a teammate. Dan Guido · ▶ 4:07

The identity shifts from I don’t need AI to I’m the one who makes AI dangerous. Dan Guido · ▶ 10:12

That’s not we use AI, that’s AI is on the team. Dan Guido · ▶ 4:33

If you want adoption, you need a safe supply chain. Basic enterprise thinking. Dan Guido · ▶ 17:59

Key Takeaways

  • Redesign the system around AI, not just the tools — giving people ChatGPT licenses without changing workflows moves nothing.
  • Reframe expert identity as ‘I make AI dangerous’ so senior engineers encode their knowledge into skills instead of resisting automation.
  • Enforce safety at the infrastructure layer — MDM-backed package cool-downs and curated allow-lists, not policy documents.