AI
Test Time Compute: Scaling Intelligence After Training
Test time compute extends a principle from training: give a model more compute, get better results. At training time, larger models over more data improve quality. At inference time, letting Claude spend more tokens on the same problem does the same thing. Anthropic’s agentic coding evaluation shows this directly: moving from Haiku to Sonnet to Opus raises scores, and then letting Opus 4.7 spend additional tokens raises scores further. The same scaling dynamic applies to agentic search, computer use, and PhD-level academic reasoning.
Effort in Practice: A Traffic Light Demo
Bleifer ran Opus 4.7 on three effort levels with the same prompt: simulate cars at a one-way street light. Low effort took 50 seconds and 4,600 output tokens. Cars stopped at the light, but the light ended up in the middle of the road. High effort doubled both time and tokens. Cars gained distinct types, the light moved to the roadside, and Opus introduced what it called an “intelligent driver model” where each car responds to surrounding traffic. Max effort ran 10x the tokens and 10x the time, returning the best graphics and most realistic driving patterns.
Three Token Types and Their Costs
Claude’s test time compute breaks into three token types. Thinking tokens are the internal monologue: chain-of-thought reasoning, scratch pad work, step-by-step planning before acting. Tool calling tokens are how Claude interacts with the world, running searches, reading files, writing code, or any of the millions of actions an agentic workflow might require. Text tokens are how Claude communicates with users, giving progress updates on long tasks, delivering summaries, or answering simple questions. All three types carry real costs in both dollars and waiting time, which is why users need controls over how Claude spends them.
Adaptive Thinking: From Fixed Sequences to Free Allocation
Early reasoning models followed a fixed sequence: think first, then call tools, then produce text. Interleaved thinking improved on this by letting Claude reason between tool calls. Adaptive thinking, the default benchmark setting since Opus 4.6, removes all ordering constraints. Claude can think before, after, or during any tool call, produce a text update mid-task, then resume thinking. For simple queries it may skip thinking entirely. Benchmarks show adaptive thinking matches or beats interleaved thinking on performance while delivering a better user experience.
Choosing the Right Effort Level
Bleifer recommends building an effort curve: chart performance against total tokens or cost across effort settings to find where diminishing returns start. Without evals, his rules of thumb: max for the hardest intelligence-demanding tasks, though the level below often matches at lower cost; extra high as the Claude Code and claude.ai default for Opus 4.7, best for coding and agentic use cases; high for general intelligence-sensitive work; medium when trading a little performance for speed; and low for short-scope or latency-sensitive tasks. Read transcripts to catch unexpected shortcuts at low effort.
Small Model or Larger Model at Low Effort?
When speed matters but the task demands real intelligence, low effort on a larger model often beats max effort on a smaller one. In the traffic simulation, Opus 4.7 at low effort used similar output tokens to Haiku 4.5 at max effort, but delivered a better result. Smaller models win on cost when tasks are bulk classification, information extraction, or basic summarization. They also produce faster time to first token. For time to last token, use a larger model at lower effort.
Notable Quotes
not how hard you want it to work. Matt Bleifer · ▶ 14:04
treating the game much like a speedrun. Matt Bleifer · ▶ 17:03
my advice would be go with extra high. Matt Bleifer · ▶ 22:56
Key Takeaways
- Enable thinking and use effort levels or budgets to modulate how much Claude thinks.
- Opus 4.7 on low effort often outperforms Haiku 4.5 on max for intelligence-demanding tasks.
- Extra high is the Claude Code default for Opus 4.7 and the best starting point for coding work.
- Use small models for fast first tokens and large models at low effort for fast final tokens.