Why Static Secrets in CI/CD Are a Liability

▶ Watch (02:03)

GitGuardian’s 2024 report found 39 million secrets leaked on GitHub. Detection took an average of 10 months. Average breach cost: $5 million. Ninety percent of those secrets were still active five days after exposure.

Four root causes drive this. Environment variables persist in pipeline logs and debug output. Shared service accounts mean one leaked credential compromises every environment from dev to production. Manual rotation fails at the pace of modern deployments. Static tokens never expire, making them, as the speakers put it, targets attackers actively seek.

Replacing Secret Zero with Workload Identity

▶ Watch (04:51)

Every Vault integration needs a secret to bootstrap authentication. That bootstrap credential is the weakest link: if it leaks, everything leaks. JWT/OIDC workload identity removes it. The pipeline job presents a cryptographically signed token carrying the repository, branch, and environment. Vault verifies it against the OIDC discovery URL and issues a 10-minute token scoped to that job only.

In the GitHub Actions Terraform demo with dynamic JWT roles (06:51), four JWT roles covered tf-plan-dev, tf-plan-prod, tf-apply-dev, and tf-apply-prod. Vault audit logs captured the actor, repository, and workflow on every request.

Right and Wrong Ways to Consume Secrets in GitLab

▶ Watch (14:15)

Using JWT authentication does not automatically make a pipeline secure. In the GitLab manual vs native secrets stanza comparison (15:46), the manual approach fetched the JWT token, called Vault’s API directly, and passed the result as an environment variable. The pipeline printed the client token and the secret in plaintext in the job log.

The fix is one configuration change: the native GitLab secrets stanza handles the entire exchange. Vault returns the secret, GitLab masks it, and even a deliberate echo of the variable prints only asterisks. Verifying the secret is correct requires checking string length, not reading the value.

Kubernetes and Jenkins: Where Secrets Still Escape

▶ Watch (18:04)

Kubernetes gives three paths to Vault secrets: the Agent Injector writes them as pod-local files, the CSI driver mounts them with auto-rotation, and Kubernetes auth uses service account tokens so no environment variable carries a credential. Binding roles per namespace and service account limits exposure when one workload is compromised.

In the Jenkins workspace secret exposure demo (19:36), the HashiCorp Vault Jenkins plugin masked the secret inside the withVault block. Written to a workspace file one line later, it appeared in plaintext. The plugin controls log output; it does not control the filesystem.

Token Lifecycle and the Three Silent Killers

▶ Watch (21:05)

Short TTLs are not enough on their own. Renewing a token outside its job scope leaves it live after the pipeline ends. A shared token across plan and apply leaves an active credential orphaned when the plan step fails. Dynamic secrets with a TTL longer than the parent Vault token become unrevocable after the parent expires.

Three silent killers: a staging token that can read production secrets, Terraform debug mode writing secrets to logs, and workspace files left on disk. The fix: one token per step, secret TTLs shorter than token TTLs, workspace cleaned at job end.

Notable Quotes

jackpots attackers do love them. So no Aditya Vardhan Pola · ▶ 4:32

Now you can look at on the screen when I Aditya Vardhan Pola · ▶ 16:37

we wanted to avoid. Always clean up your Aditya Vardhan Pola · ▶ 20:52

Key Takeaways

  • Use JWT/OIDC workload identity to authenticate CI/CD jobs without storing any bootstrap credentials.
  • Scope each Vault JWT role to a single pipeline action and environment, with TTLs of 10-15 minutes.
  • Always clean up workspace files after a job; masked secrets in logs can still be plaintext in artifacts.