AI
Deploying Terraform Enterprise on Nomad
Benjamin Lykins recommends Nomad over Docker or Podman for self-managed TFE because Nomad’s dispatch model separates the long-running control-plane job from short-lived agent batch jobs that tear down after each run. The Nomad UI TFE agent job walkthrough (04:40) shows the control plane spread across availability zones with agents on separate CI/CD node infrastructure.
Each dispatched run gets a dynamic single-use token that cannot authenticate again once used. Only that one run communicates back to TFE from Nomad, eliminating any shared-credential risk across concurrent runs.
Custom Agents, Hooks, and Dynamic Credentials
Custom agent images built with a Dockerfile let teams inject binaries and run them at four lifecycle points: pre-plan, post-plan, pre-apply, and post-apply. For internally developed Terraform providers, hooks exchange a JWT workload identity token at pre-plan for valid credentials, replacing long-lived secrets in environment variables.
Terraform’s ephemeral resources and write-only arguments take this further. The root Postgres password used to configure a Vault secrets engine is never written to state. Rotate it after apply and it stays out of every subsequent plan.
CI/CD Pipeline Design for Enterprise Scale
Passing cloud credentials as environment variables into a pipeline run disables HCP Terraform’s health checking and drift detection, both of which need credentials available outside a pipeline invocation. Dynamic provider credentials fix both problems. For pipelines making direct API calls, HCP Terraform caps requests at 30 per second per identity, so concurrent runs sharing one team token hit that ceiling fast. HashiCorp’s purpose-built TFCI tool handles call sequencing and avoids that limit.
Dependabot and Renovate both integrate with the private registry in HCP Terraform and TFE, triggering speculative plans against affected workspaces when a new module version publishes.
Governance: Per-Resource Policy Over Module Lock-In
Module-based governance breaks at scale. Users cannot write resources outside approved modules, module-owning teams bottleneck under change requests, and enforcing a new control requires every workspace to adopt a new module version first.
Per-resource Sentinel policies remove that dependency. Compliance checks run against each resource on each run, regardless of what module produced it. Push a new policy and it enforces immediately across every workspace. Users can fork modules or write from scratch, and compliance stays consistent either way.
Observability by Default and Disaster Recovery
Every resource Terraform provisions in production should get a monitor in the same run. Tags make routing automatic: pulling the workspace name as a built-in tag plus CI/CD metadata like repository and commit lets you trace any AWS resource back to the workspace that owns it.
“essentially, the more tags, the better.” (Benjamin Lykins)
For self-managed TFE, the AWS step-function DR failover demo (26:46) shows RDS failing over from US-East to US-West-1, DNS cutting over, and the failover autoscaling group coming up healthy in under 5 minutes.
Notable Quotes
essentially, the more tags, the better. Benjamin Lykins · ▶ 22:59
organizations on the face of the earth. John Weigand · ▶ 1:34
strongly opinionated user-friendly flow. John Weigand · ▶ 15:41
Key Takeaways
- Run TFE on Nomad with separate control-plane and agent node groups, single-use tokens per run.
- Sentinel policies should target individual resources, not modules, so compliance updates deploy instantly.
- Tag every provisioned resource with workspace, repo, and commit metadata; enforce it via Sentinel.