Real-Time Captioning for Fans Who Need It Most

▶ Watch (04:38)

The Pacers Sports & Entertainment Group set out to serve fans with hearing impairments and non-native speakers at Gainbridge Fieldhouse. Working with Microsoft, they built a real-time speech-to-text pipeline using Azure AI Speech and Azure AI Foundry, training custom models on hundreds of hours of Pacers and Fever game calls. The result: caption accuracy climbed from 91% to nearly 99%, and response time dropped

“a week has become two minutes” — Jared Chavez

for audio processing. Live captions now appear on arena boards and push to the Pacers app in real time, supporting 12 languages with 97 available.

Natural Language Queries at the NFL Combine

▶ Watch (11:46)

NFL scouts evaluating 330 draft-eligible players had no way to filter drill stats without pulling multiple reports. Three weeks before the 2025 Combine, Southworks built a natural language agent on the existing Combine app: Azure OpenAI GPT for reasoning, Cosmos DB for storage, Container Apps for hosting. Scouts ask “give me the fastest 40 times of a defensive lineman” and get results in

“1.2 seconds.” — Johnny Halife

He walked through the Combine Copilot live demo (16:26), showing the agent expand a brief follow-up into the full drill query, then render a bar chart via a second tool call.

Scaling Infrastructure for Event-Driven Workloads

▶ Watch (38:33)

Both teams describe the same problem: event-driven traffic means the system sits idle for hours, then gets hit with scouts or fans all at once. Halife calls his solution “the holy trinity”: Azure OpenAI PTUs for constant, predictable token throughput between drill reps, Container Apps with pre-provisioned capacity to eliminate cold starts, and Cosmos DB autoscaling to absorb data spikes. The Pacers deal with optical tracking bursts that jump from two gigabytes to four terabytes in a single batch, handled with Azure Databricks, Spark, and Autoloader.

Model Lifecycle and Rollback in Production

▶ Watch (43:16)

The Pacers’ caption model started the season with a 13% error rate on their test set. After four training iterations, it sits at 1.14%. The baseline Azure Speech model alone dropped from 13% to 5% in a recent update. Chavez’s team keeps all previous model versions live. When a new version underperforms, the broadcast team changes the endpoint in Foundry without touching the backend code. The goal for this season: retrain a model by 9 a.m. and have it deployed before tip-off at 6 p.m. same day.

Storing Agent Thread History in Cosmos DB

▶ Watch (52:40)

Azure AI Foundry Agent Service now stores agent thread data directly in Cosmos DB via a BICEP deployment template. James Codella showed the Cosmos DB agent thread storage demo (52:40), querying a live agent playground and then opening Cosmos DB to show the corresponding JSON documents: thread messages, tool call records, and model outputs stored across three containers. The integration gives teams two things: persistent context for multi-turn agent conversations, and a queryable log for identifying where agents need retraining.

Notable Quotes

a week has become two minutes Jared Chavez · ▶ 9:56

1.2 seconds. Johnny Halife · ▶ 24:56

I think I’ll call it query entropy. Johnny Halife · ▶ 34:05

Key Takeaways

  • Azure AI Speech trained on hundreds of hours of Pacers game calls cut caption error rates from 91% to 99% accuracy.
  • Southworks built the NFL Combine Copilot agent in three weeks, returning scout queries in 1.2 seconds.
  • Single-endpoint model deployment lets broadcast staff roll back to a previous model version without touching code.