The Problem With Frontier and Local Models

▶ Watch (01:25)

Frontier models from OpenAI, Google, and Anthropic refuse to write malware. Even when they cooperate, submitting private research to their APIs creates privacy exposure, and token costs compound fast in automated pipelines. Hosting an open-source alternative like DeepSeek R1 requires hundreds of thousands of dollars in GPUs. Consumer hardware runs Llama and Qwen fine, but those models can’t handle highly technical work. For anyone running automated red team workflows, neither option works.

Model Size vs. Skill Count: The Specialist Tradeoff

▶ Watch (03:22)

Avery’s framework: quality per skill roughly equals model size divided by skill count. A generalist like ChatGPT spreads parameters across recipes, emails, and code. A small model trained on fewer tasks concentrates the same per-skill capacity. Chain-of-thought prompting demonstrated the gap: Google’s Palm 540B jumped from 18% to over 50% on grade-school math with no additional training. OpenAI’s o1 pushed further, using reinforcement learning to move from under 10% to over 70% on the AIME benchmark. Fewer skills, better specialist.

Reinforcement Learning With Verifiable Rewards

▶ Watch (15:46)

RLHF needs humans to rate outputs, then a separate reward model to estimate scores. RLVR cuts both steps. A program, the verifier, evaluates each output directly. For a multiple-choice question, it parses the model’s answer and checks it against a key. No human rating, no reward model. DeepSeek’s GRPO algorithm made this practical for general LLMs, following what OpenAI did with o1. The model generates a chain-of-thought answer, the verifier scores it right or wrong, and every reasoning step that led to a correct answer gets rewarded.

Why Malware Development Fits Verifier’s Law

▶ Watch (20:27)

Jason Wei at OpenAI defines five attributes of a verifiable task: objective truth, fast verification, scalability, low noise, and a continuous reward. Malware evasion hits all five. Fewer alerts, lower severity, is always better, that is the objective truth. Virus Total-style sandboxes verify automatically, with no human in the loop. Samples run in parallel on AWS or Azure. Training against the same EDR products used in real operations keeps noise low. And the reward is continuous: one low-severity alert scores higher than one high-severity alert, and no alerts scores highest.

Building and Benchmarking Dante

▶ Watch (21:57)

The verifier runs in two stages. A static sandbox confirms the PE calls back to a Cobalt Strike team server, so the model can’t earn reward from a benign binary. A fresh domain-joined VM then installs Microsoft Defender for Endpoint, detonates the loader, and polls the alert API. Dante, based on 7B Qwen 2.5 Coder, cost $250 for 13 hours of SFT and $1,100 for 56 hours of RLVR. The Dante shellcode loader demo (29:35) shows the full generation UI. Single-shot, it writes fully evasive loaders 8% of the time. Pass-8 pushes that past 30%.

Notable Quotes

First off, there’s an objective truth. Kyle Avery · ▶ 20:37

think it’s clear that that lowcost Kyle Avery · ▶ 30:10

the sky is the limit but then it goes on Kyle Avery · ▶ 7:44

Key Takeaways

  • A 7B specialist model trained with RLVR outperforms DeepSeek R1 at writing evasive shellcode loaders.
  • Malware evasion meets all five attributes of verifier’s law: objective truth, speed, scale, low noise, continuous reward.
  • SFT plus 56 hours of RLVR on eight H100s cost roughly $1,350 total; the resulting model runs on a consumer 3090 or 4090.