AI
How Reasoning Models Work and What They Can Do
o3 and o4-mini are reasoning models that write their work into a scratchpad before producing output. That scratchpad, called chain-of-thought, lets the model backtrack and self-correct. The result: o3 scores 20 out of 100 on Humanity’s Last Exam, a benchmark designed to defeat AI, and currently tops the leaderboard.
“Now, o3 gets 20 out of 100.” — Keiji Kanazawa
Nick Brady showed the chain-of-thought summarization demo (26:09) live, revealing the model’s internal reasoning summary for the first time on Azure. o4-mini costs less and beats o3 on the AIME 2024 math benchmark.
Pharma Regulatory Docs in One Week Instead of Three Months
Deep Intelligent Pharma used o1 in December 2024 to auto-write a clinical-trial protocol for Kobe University. Three days after delivery, the hospital team reported zero edits were needed. A protocol that takes a human expert three months was done in under a week.
“This was impossible, for example, with GPT-4o,” — Shinya Yamamoto
A second case at Osaka University Hospital produced three scenario options for a novel drug’s clinical endpoint, again in under a week. Yamamoto called it the first case in the world where gen AI wrote a clinical-trial protocol without any revision by a human expert.
GPT-4.1 Family: 1M Token Context and Code Editing
GPT-4.1, 4.1-mini, and 4.1-nano carry a one-million-token context window, eight times the previous limit. Nick Brady loaded the GPT-4.1 1M-token Hanselminutes demo (42:26) with 989,000 tokens of podcast transcripts and asked a zero-shot question about a 2006 Grease Monkey script. The model answered correctly.
The team’s selection advice: start with GPT-4.1 or 4.1-mini, then move to o4-mini or o3 only if those fall short. GPT-4.1-mini is already replacing 4o at lower cost for some customers. The models also output surgical code diffs rather than full file rewrites.
Real-Time Audio API and the Deep Research API Announcement
Azure OpenAI now ships WebRTC and WebSocket support for the GPT-4o real-time audio model. Nick Brady ran the real-time audio tool-calling demo (50:26) live, with the model fetching Seattle weather, converting Celsius to Fahrenheit, and storing session memory, all via natural-language tool descriptions over a streaming audio connection.
“We’ve now just recently shipped WebRTC and WebSocket support” — Nick Brady
The session closed with the main announcement: the Deep Research API, powered by Bing Search. Deep research exists in ChatGPT today but has no API. Steve Sweetman told attendees they could sign up to be first in line, with the API shipping within two to four weeks.
Q&A
With so many models available in Foundry, do developers need to pick the right one manually, or is there something that handles that automatically? Sweetman announced Model Router, shipping that week, a pre-trained model that automatically routes between GPT-4.1, 4.1-mini, 4.1-nano, and o4-mini based on prompt complexity. ▶ 58:43
Notable Quotes
Now, o3 gets 20 out of 100. Keiji Kanazawa · ▶ 11:38
This was impossible, for example, with GPT-4o, Shinya Yamamoto · ▶ 18:27
We’ve now just recently shipped WebRTC and WebSocket support Nick Brady · ▶ 53:41
Key Takeaways
- o3 outscores every model on Humanity’s Last Exam at 20 out of 100; o4-mini beats o3 on the AIME 2024 math benchmark.
- Deep Intelligent Pharma cut clinical-trial protocol writing from three months to one week with zero human revisions using o1.
- The Deep Research API, backed by Bing Search, was announced at Build 2025 with a two-to-four-week ship window.