Four Governance Gaps That Exposed Synchrony’s Vault Blind Spots

▶ Watch (02:12)

Synchrony manages 70 million active accounts and $180 billion in purchase volumes. Four recurring failures made secrets governance untenable: no centralized view of what was stored or when it last rotated, inconsistent practices across teams, manual SOC investigation for every compliance question, and developers discovering their 12-month AppRole secret IDs had expired only after a pipeline broke or a production login failed.

Ninety percent of internal stakeholders flagged the manual investigation problem. The Vault team had no authority to inspect secret values, so every inquiry became an escalation chain that sometimes ended in an outage.

How Vault Namespaces and Splunk Logs Became One System

▶ Watch (08:41)

Synchrony runs a hybrid cloud environment, so on-prem and cloud application teams both needed consistent access to Vault. The team structured that access with namespaces, giving each application team logical isolation and ownership over their own secrets. Vault audit and operational logs were already flowing into Splunk but going unused. The fix was to repurpose those logs, add metadata fields at onboarding (primary owner, secondary owner, team distribution list, secret type), and build dashboards on top.

A governance feedback loop closed the design: when dashboards revealed a pattern, the team updated policy rules directly from that output.

Three Splunk Dashboards That Enforced Rotation Without Lifting a Finger

▶ Watch (12:00)

Three Splunk dashboards replaced the manual work. The static secret expiry dashboard reads the secret type each team classified at onboarding (API key, certificate, database password), applies the matching rotation window, and fires alerts at 60, 30, 20, and 10 days out. Shiva walked through the static secret expiry dashboard walkthrough (16:10) live, including a secret 412 days past its one-year window. The AppRole dashboard tracks accessor IDs the same way. Siva demonstrated the AppRole secret ID expiry dashboard walkthrough (21:55), where one ID had six days left and had already received escalating emails since day 60.

From Firefighting to Proactive Governance: Outcomes and What Comes Next

▶ Watch (30:31)

The three dashboards delivered concrete numbers. Audit teams reported an 80% reduction in compliance toil. The static secret expiry tracker reached 100% flagging before expiry, and 80% of those alerts were resolved by developers without involving security or operations teams. AppRole escalations dropped to zero. App owners gained a self-service model, audit teams pulled quarterly evidence directly from Splunk, and leadership saw fewer escalations land on their desks.

The roadmap replaces email alerts with ServiceNow incidents tied to configuration item IDs, adds Teams and Slack notifications, and integrates the PKI secret engine for internal certificate rotation.

Notable Quotes

we saw an 80% reduction in compliance Siva Siraparapu · ▶ 14:42

we saw 100% of the secrets were getting Siva Siraparapu · ▶ 19:05

we had zero escalations because an app Siva Siraparapu · ▶ 24:24

Key Takeaways

  • Routing Vault audit logs to Splunk and adding metadata fields gave Synchrony a single source of truth for 70M-account-scale secrets governance.
  • Three automated dashboards eliminated 80% of compliance toil and reduced AppRole expiry escalations to zero.
  • Requiring teams to classify secrets by type at onboarding lets rotation policies apply automatically, with no manual tracking.