AI
Three Archetypes of AI Applications
Three patterns are forming for AI applications. The AI frontend puts a chat interface between the user and the LLM, sometimes embedded in an existing tool like VS Code with GitHub Copilot. The AI backend replaces a business logic component with an LLM call, and the end user never knows it; Tinderholt’s own team at Microsoft built one. The AI workflow pattern, which this talk focuses on, feeds external data sources into the LLM to contextualize its responses. All three patterns add new services, new data planes, and new configuration surfaces on top of everything teams already operate every day.
RAG Architecture on Azure: Gather, Chunk, Query
RAG has two sides. On the gather side, documents arrive as PDFs; text goes to Azure Document Intelligence for chunking, images go to Azure Vision. Everything lands in Azure AI Search, chunked and correlated so the LLM can pull the most relevant pieces at query time. On the ask side, a React frontend talks to a REST API on Azure Container Instances, which routes to AI Search and then to Azure OpenAI. The AI services sit on the same ARM control plane but add configuration ARM alone cannot manage, including model deployments, prompt libraries, and responsible AI policies.
Terraform Stacks as the IaC Answer to AI Complexity
Terraform Stacks splits the RAG deployment into four components: network, shared infrastructure (container registry, observability), AI services, and application hosting. Each component is a root module. Dependencies between them are first-class HCL citizens, not bailing wire stitched together in GitHub Actions. A change to the AI layer shows precisely what, if anything, changes in the network or application layers. Tinderholt used federated identity credentials throughout, so no secrets appear in the configuration. The multi-provider support means the ARM control plane and the Azure AI data plane both sit under Terraform without forcing everything into one root module.
Live Demo: Scalpel-Level Changes Across the Dependency Graph
Tinderholt pulled up HCP Terraform live, one day after Stacks GA shipped and briefly broke his demo. He opened the AI components stack, edited a model capacity variable in VS Code to resolve token-limit errors, committed directly to main, and watched Stacks pick up the change automatically. The model capacity update via Terraform Stacks (22:49) shows Stacks running a plan, confirming only the AI component changed, then applying. Network and application stacks showed no changes. One-line config edit. No downstream impact.
“No drift is happening here.” — Mark Tinderholt
Notable Quotes
This is completely secretless, Mark Tinderholt · ▶ 22:20
I’m in dev, so I can get some fast feedback Mark Tinderholt · ▶ 23:44
No drift is happening here. Mark Tinderholt · ▶ 25:03
Key Takeaways
- AI applications need four infrastructure layers: network, shared services, AI data plane, and application hosting.
- Terraform Stacks makes cross-component dependencies explicit and limits blast radius to the changed layer only.
- AI configuration, such as prompt libraries and model deployments, belongs in version-controlled IaC, not click-ops.