Why Bare Prompts Fail for Infrastructure Code

▶ Watch (02:49)

LLMs have no idea about your project’s purpose or your existing infrastructure. Ask one to “generate an S3 bucket in Terraform” and it draws from public blog posts, not your actual repo. The fix is context: RAG pulls local files and remote sources into the prompt. Modern AI assistants go further, reading repo structure, function dependencies, and git history. If you tried an AI coding assistant 16 months ago and gave up, the context problem has largely been solved. Try again.

Tool Execution and MCP Close the Hallucination Loop

▶ Watch (07:51)

Even with good context, LLMs hallucinate. The correction loop: tell the LLM to call Terraform directly, run validate, plan, and apply, then report back. What used to require a human clicking through each step becomes a tool call the model handles itself, with you only reviewing the final output. MCP standardizes this across any LLM and any tool.

“think about it as a USB for your LLM.” – Welly Siauw

The Terraform registry, AWS docs, internal wikis, and observability APIs all become callable. Prompt quality becomes the engineering artifact that matters.

Multi-Agent Orchestration for Infrastructure at Scale

▶ Watch (13:24)

The AWS cloud control provider auto-generates from AWS schemas. New resources ship with no human-written code examples, and documenting each one took engineers one to two days per resource. TANGO (Terraform AI Native Generator and Orchestrator), built by an AWS intern named Ekaterina, replaced that process. An orchestrator coordinates four sub-agents: discovery (queries GitHub and DynamoDB), documentation (queries Terraform registry and AWS docs), a Terraform agent (writes and tests code), and a validation agent, a separate LLM that re-runs plan and apply to grade the first agent’s work.

“we have generated over 500 uh resources” – Welly Siauw

Average cost: $5 per resource.

LLM-as-Judge: Validation That Blocks Optimistic Hallucinations

▶ Watch (18:42)

The demo targets EC2 Network Insights Analysis, a resource published a few weeks before the talk. The documentation agent fetches AWS docs and the Terraform registry schema, then hands both to the Terraform agent. That agent writes configurations, self-reflects, and runs the TANGO live demo (18:42): validate, plan, apply. When apply fails on a missing dependency attribute, the agent reads the error and retries. A second LLM then reruns plan and apply independently to confirm success, checking variable naming conventions against org standards before flagging the result as approved.

Notable Quotes

Your prom is your intellectual property. Welly Siauw · ▶ 24:49

think about it as a USB for your LLM. Welly Siauw · ▶ 12:13

we have generated over 500 uh resources Welly Siauw · ▶ 17:07

Key Takeaways

  • Context from RAG and repo structure is what separates useful AI coding assistants from useless ones.
  • Tool execution (validate, plan, apply) lets the LLM self-correct without human hand-holding at every step.
  • A second LLM grading the first agent’s output prevents optimistic hallucinations from reaching code review.