AI
The Model Catalog: 11,000 Models and How to Navigate Them
Azure AI Foundry’s model catalog now lists 11,000 models. The problem it creates is also the reason for the session: how do you pick? Moneypenny mapped the catalog to four archetypes. Foundation models handle almost any task if you prompt them enough. Generative models communicate well but lack grounding. Retrieval-augmented models plug into external data. Reasoning models, like DeepSeek, think before answering.
“Big models are great for generalizing,” — Naomi Moneypenny
The catalog spans Foundry Labs (experimental), industry-specific models, direct-from-Microsoft scale models (OpenAI, Grok 3, Mistral, Meta Llama), and a long tail of open-source and partner models. Picking the wrong archetype wastes both tokens and time.
Model Selection Tools: Leaderboards, Benchmarks, and Trade-off Charts
Foundry’s model leaderboard aggregates industry-standard benchmarks into a single quality index, then surfaces cost and throughput rankings alongside it. Alex walked through the trade-off charts live: low cost and high quality puts DeepSeek R1 and o3-mini in the most attractive quadrant. For coding specifically, o4-mini scores highest on the benchmark leaderboard.
The compare tool lets developers select multiple models, overlay their metrics, and switch between them in the playground before committing to a deployment. No separate tooling or spreadsheet required.
Flexible PTUs: Shared Capacity Across Model Families
Provisioned Throughput Units (PTUs) are now flexible across all direct-from-Azure models, not just Azure OpenAI. In the PTU capacity calculator demo (19:44), Alex entered 20 peak calls per minute, 2,000-token prompts, and 1,000-token completions, then got a recommendation of 50 PTUs for GPT-4o. He allocated 100 PTUs to DeepSeek from the same 300-PTU pool. Both deployments ran from a single quota screen.
An agent routing some tasks to GPT-4o and others to DeepSeek no longer needs separate capacity contracts. Standard token-based consumption stays available for evaluation and lighter workloads.
Hugging Face and NVIDIA NIMs: Open-Source at Enterprise Scale
Foundry now surfaces over 10,000 trending Hugging Face models, scanned for vulnerabilities and exposed via OpenAI-compatible APIs. Text generation and embedding models drop into existing inference SDKs. The roadmap adds 3D generation, video generation, and time-series forecasting, with day-one access to new releases.
NVIDIA NIMs are pre-built containers with CUDA, TRT-LLM, and Triton optimizations baked in. The NVIDIA NIMs deploy wizard demo (35:50) showed Llama 3.3 Nemotron Super deploying on an NC H100 VM at $1 per GPU per hour after a 90-day trial. Epic reported lower latency and faster batch processing of clinical notes from its initial deployment.
Epic: Agents on 300 Million Patient Records
Epic went live with its first generative AI use case in April 2023, drafting responses to patient messages. By the session date, 120 use cases were deployed across 374 groups globally. Drew McCombs cited The Christ Hospital in Ohio: it diagnoses stage one and two cancer from secondary radiology findings at 69%, against a 46% US average.
The Epic Sidekick research agent demo (46:28) ran against Cosmos, a research database holding 300 million de-identified patient records and 17 billion clinical encounters. A researcher narrowed that to roughly 8,000 patients, broke down by age and demographics, and generated a shareable summary. Azure’s regional deployments let Epic process data in-country to meet privacy regulations.
Saifr: Financial Compliance Agents Built in Hours
Saifr, born inside Fidelity Labs, trains models on financial regulations to help compliance officers handle AI-generated content. Four Saifr models sit in the Foundry catalog. The compliance agent uses GPT-4o mini as an orchestrator and two Saifr specialist models as tools via function calls.
In the Saifr compliance agent live demo (56:35), the agent received “Invest in bitcoin to get rich fast” and two other non-compliant sentences. It called the compliance tool, flagged each violation against FINRA 2210, called the language suggestion tool, and rebuilt the paragraph. David Johnston said the entire build took a few hours: a prompt and two OpenAPI 3 tool definitions.
Notable Quotes
Big models are great for generalizing, Naomi Moneypenny · ▶ 2:38
AI is now table stakes, Tina Manghnani · ▶ 33:20
The prompt was – worked first time. David Johnston · ▶ 1:03:38
Key Takeaways
- The Foundry catalog now has 11,000 models, with leaderboards and trade-off charts helping developers balance quality, cost, and throughput.
- Flexible PTUs let enterprises share a single capacity pool across GPT-4o, DeepSeek, and other direct-from-Azure models in one agent.
- Epic’s 374 production deployments show measurable clinical impact; Saifr built a compliance agent with two tool calls in a few hours.