AI
Why GreyNoise Built Orbee: Scale That Broke the Team
GreyNoise runs a fleet of sensors across 90-plus countries. In the last 90 days alone, that grid generated 22 TB of packet captures, 32 TB of session data, and 20 million Elasticsearch documents. One small team was responsible for finding signal in all of it. When ReactShell hit in December 2025, the volume became impossible. By January 6th, the team had observed the ReactShell payload storm stats (02:51): 8,200 unique IPs from over 1,000 ASNs, slinging 70,000 completely unique payloads, some generated with Claude.
“we scaled our fleets, but the scale is what broke us. So we built Orbee.” — Glenn Thorpe
Orbee’s Skill System: Teaching the Agent Domain Knowledge
Orbee starts with an empty head every session. For threat hunting, that makes it about as useful, in Rudis’s words, as “a 13-year-old with access to both a flamethrower and Wikipedia.” The skill system is the fix. Each skill is a structured system prompt, bundled with resources, tools, SQL, Bash, and Go snippets, that loads a specific domain into Orbee before it starts work. Skills cover tag triage across 3,500 detection rules, pcap analysis, out-of-band OT domains, Census enrichment, markdown formatting, and findings validation. The model doesn’t figure out the methodology — the human does, then encodes it.
“Bob is smart, but Orbeez is fast, working all the time.” — Glenn Thorpe
MCP Tools Arsenal: Ground Truth Data at Orbee’s Fingertips
Orbee connects to a dozen MCP servers. The core stack: Archammy and the GreyNoise API for ground-truth honeypot data, Census for turning IPs into infrastructure profiles, JA4 fingerprints for connecting attacker nodes, DuckDB for analytics, t-shark with 30 years of pcap knowledge for packet work, and Feedly CTI for external context. Census found the Canadian hosting cluster credential-stuffing SonicWalls: 2,000 nodes, 70 days running, data now with the CCCS. Rudis on the analytics choice:
“friends don’t let friends let LLM do their own analytics because they suck at everything if you don’t give them tools.” — Bob Rudis
Mid-talk, the Orbee live threat hunt check-in at phase 4 showed it had already queried Census, VirusTotal, and JA4 and was writing the final report.
Preventing Failures: Guard Hooks, Decision Trees, and Bulletproof Patterns
“Safety is prevention and validation as detection.” — Bob Rudis
Guard hooks are shell commands that terminate bad behavior the moment it starts, before it reaches output. Decision trees enforce step order: no dessert before dinner. Bulletproof patterns eliminate ambiguity on routine tasks — timestamp conversions have one correct format, and Orbee is told exactly what it is. Checklists let you verify coverage without assuming the model did what you expected. These controls constrain failure modes up front, but Thorpe is direct: guardrails don’t prevent crashes. They only change what happens after one.
Detecting Failures: The Validator and Human-in-the-Loop Review
Guardrails stop some failures. They don’t catch the ones that slip through anyway, so Thorpe built a validator: a second model that takes a finished Orbee report, extracts every claim, and re-queries the data sources to confirm each one. Validator findings on early Orbee reports were alarming — 78 claims extracted, fewer than half verified. The model had been summarizing tag names across a long run, and by the end had invented names that don’t exist in GreyNoise’s detection library. The fix went straight back into the skill: never paraphrase or alternate tag names. That feedback loop is the point. A human still reads the final output before it reaches any stakeholder. It’s your company and your reputation, and a self-driving car crash is still your problem.
“assume these things are going to do something really stupid and plan around it” — Bob Rudis
Outcomes and Open Source Gifts
Orbee has been filing real tickets. Over roughly a month and a half, it opened Linear issues each time it found something actionable — including a GitHub takedown notice after detecting malware hosted there. Rudis filed it himself, with the exact language Orbee drafted.
Three tools ship free with this talk: a Census MCP skill for infrastructure enrichment, a Suricata MCP server for iterating on detection rules and pcaps, and an O-domain command-line tool and MCP server. All are in the repo.
Q&A
Everybody seems to be using Claude Code to orchestrate agents. Have you thought about or tried using something like LangGraph or LangChain instead? They’ve already hit Claude Code’s limits and plan to build their own threat-hunting agent framework in Go, but recommend others start with Claude Code and migrate when they’re ready. ▶ 24:08
When do you outgrow Claude Code — what are the limitations? Context window cost is the main driver — Bob doesn’t want to pay for million-token windows — and Orbee at v0.93 will move to a new framework after 1.0 when it needs to pull in additional data sources. ▶ 25:20
Notable Quotes
we scaled our fleets, but the scale is what broke us. So we built Orbee. Glenn Thorpe · ▶ 5:26
Bob is smart, but Orbeez is fast, working all the time. Glenn Thorpe · ▶ 9:50
friends don’t let friends let LLM do their own analytics because they suck at everything if you don’t give them tools. Bob Rudis · ▶ 14:36
Safety is prevention and validation as detection. Bob Rudis · ▶ 15:38
assume these things are going to do something really stupid and plan around it Bob Rudis · ▶ 22:20
Key Takeaways
- Build domain-specific skills as structured system prompts — LLMs start empty and need loaded context to be useful.
- Use deterministic tools for analytics; never let the LLM do its own math or data queries without DuckDB or equivalent.
- Run a second model as a validator to extract and recheck every claim before any report reaches a human stakeholder.