AI
Shared Staging Can’t Serve 200 Engineers
Trivago runs 200+ engineers simultaneously, with more than 100 active frontend PRs and 100 backend PRs on any given day. Shared staging blocks progress: two engineers’ PRs interfere with each other, making test results unreliable. Local environments don’t match production. The failure case was concrete: a developer merged code that passed local tests, introduced an incident, and cost €100k. Preview environments exist to catch that class of bug before merge.
One Service Per PR, Wired to Staging
Opening a PR triggers an automatic deployment. Only the changed service gets built and deployed. Its dependencies (search, payment) stay on the shared staging environment. The developer gets a URL to share with QA or stakeholders. Integration tests run against the live deployment. When the PR merges, the environment tears down automatically. Idle environments scale to zero. This gives each of 200+ developers an isolated test surface without waiting for a shared slot.
The Cost Problem: 600 Environments on Kubernetes
600 simultaneous preview environments at two pods each, averaging 1 CPU and 1 GB RAM per pod, would exceed production infrastructure cost. Cron-based scaling doesn’t work because engineers have flexible hours. Kubernetes 1.36 supports scale-to-zero but not HTTP-triggered activation. Previously, Trivago’s frontend previews ran on Google Cloud Run, handling 600 environments for about €100 per month. But Cloud Run meant two separate monitoring and deployment stacks. The goal: unify everything on GKE and achieve comparable per-environment cost with native tooling.
KEDA HTTP Add-on: Queued Requests, Not 503s
KEDA is a CNCF graduated project written in Go that extends Kubernetes autoscaling. Its HTTP add-on routes all traffic through an interceptor. When a pod is at zero, an incoming request gets queued in the interceptor, which triggers a scale-up and releases the held requests once the pod is ready. Users wait through the cold start but don’t receive 503 errors. For Trivago’s frontend services, pods come up in under 30 seconds. The add-on also supports header-based routing, giving ingress users gateway API-style traffic splitting without migrating.
ArgoCD ApplicationSets Close the Loop
Each PR triggers a GitHub Actions workflow that builds a Docker image and renders Kubernetes manifests using Helm and Skaffold. Those manifests push to a Git branch. ArgoCD ApplicationSet picks them up through two generators: a PR generator watching for pull requests with a specific label, and a Git generator. ArgoCD deploys into a dedicated namespace. Incoming requests route through KEDA’s HTTP interceptor. The entire lifecycle is automatic: environment created on PR open, torn down on PR merge, and scaled to zero when idle.
Q&A
How do preview environments handle test data for stateful features? Most Trivago previews are stateless; services read from existing databases without writing, so no per-environment data setup is required. ▶ 26:45
Does the browser hang while a pod starts from zero? The KEDA interceptor queues the request until the pod is ready; for Trivago’s services, startup takes under 30 seconds, and CI waits for pod availability before running integration tests. ▶ 28:25
Why is GitHub Actions needed at all if ArgoCD handles deployment? Building the Docker image and rendering Helm manifests requires CI; Trivago also intentionally mirrors the production pipeline so engineers can test infrastructure changes through the same path. ▶ 30:13
Does Trivago use KEDA in production beyond preview environments? Yes, with 60+ Kafka consumers scaled by the KEDA Kafka scaler, reducing idle resource cost when messages arrive irregularly. ▶ 32:51
Notable Quotes
it needs to proxy all of your traffic. Jan Wozniak · ▶ 11:48
Completely terrible. Armin Aminian · ▶ 14:11
getting rid of the HTTP scaled object. Jan Wozniak · ▶ 16:10
unified platform no cloud run anymore. Armin Aminian · ▶ 25:31
Key Takeaways
- KEDA’s HTTP add-on queues requests during cold starts, preventing 503s without requiring client-side retry configuration.
- ArgoCD ApplicationSet PR generators automate environment creation and deletion as pull requests open and merge.
- 600+ Kubernetes preview environments become cost-effective only with scale-to-zero; always-on pods would exceed production costs.
About the Speakers
Armin Aminian is a Site Reliability Engineer at trivago with more than five years in cloud computing and a background in network engineering. He focuses on the CNCF ecosystem to build platforms that improve developer productivity.
Jan Wozniak is a software engineer at Kedify and a maintainer of KEDA. He has contributed to multiple parts of the Kubernetes ecosystem and maintains the KEDA HTTP add-on alongside Red Hat contributors.