Choosing Where to Run Vault

▶ Watch (03:29)

Bare metal gives Vault the smallest attack surface because there is no abstraction layer between the process and the hardware. VMs cost less to provision and add one layer. Kubernetes adds more, requires dedicated expertise, and widens the surface further. Cloud deployments trade sovereignty for simplicity: one-click cluster installs and managed hardening come at a higher per-unit cost.

Pick the platform your team can actually operate. Krass recommends an odd-node cluster (typically three or more) and a disaster recovery secondary for fault tolerance across data centers.

Auto-Unseal and TLS Hardening

▶ Watch (07:20)

Three auto-unseal paths exist: a second Vault using its transit engine, a cloud KMS, or an HSM. Cloud KMS requires internet access and stores cryptographic material outside your perimeter. Krass’s recommendation is the HSM, a dedicated piece of hardware plugged directly into the server, because it limits access to local connections only.

“get your certificate right” — Leon Krass

On TLS, Krass saw many customers struggling with certificate mismanagement. Put the cluster address in the SAN field and individual node addresses alongside it so one certificate works across all nodes. Automate renewal. Avoid transparent proxies that inspect traffic targeting Vault.

Deployment Automation

▶ Watch (15:04)

For on-premises installs, Krass points to the Ansible Vault role. Write a playbook around it, set the inventory and variables, run Ansible, and the cluster is up. SVA maintains the role and accepts pull requests. For cloud, the HashiCorp Terraform provider handles Vault deployment, and hyperscaler-specific providers released recently install all required resources in a single apply.

Raft is the recommended storage backend in almost every case. It was built for Vault, performs well, and Krass has never hit an issue with it. The one exception: if write throughput is a bottleneck, Consul can increase it.

Backup, Observability, and Troubleshooting

▶ Watch (18:32)

Use the official Vault Terraform provider to configure everything inside Vault as code. That gives you reviewable changesets and GitOps workflows. On backups:

“a snapshot is actually not a backup” — Leon Krass

A Vault snapshot captures data, but you also need the infrastructure-as-code to rebuild nodes before restoring it. Test the restore before an incident forces you to. For observability, ship operational and audit logs to Loki, Splunk, or Elastic. Feed metrics through the telemetry stanza to Prometheus. Add synthetic monitoring: scripted login, KV write, KV read, token revoke, run continuously.

Notable Quotes

get your certificate right Leon Krass · ▶ 9:51

a snapshot is actually not a backup Leon Krass · ▶ 19:46

vault is all about security of course Leon Krass · ▶ 9:33

Key Takeaways

  • Run Vault on bare metal or VMs for the smallest attack surface; Kubernetes adds abstraction layers that widen it.
  • HSMs give local-only cryptographic material access; cloud KMS works but puts unseal keys outside your perimeter.
  • A Vault snapshot needs accompanying infrastructure-as-code to be a full recovery plan; test restores before you need them.