AI
KCP as a Multitenancy Control Plane for Kubernetes-Style APIs
KCP is a standalone API server that speaks the Kubernetes resource model but is not Kubernetes. It shares libraries with the kube-apiserver and responds to kubectl requests, but runs no workloads. Its purpose is to serve hundreds or thousands of logically separated API endpoints from a single instance, acting as a control plane above existing Kubernetes clusters. Platform teams use it for internal developer platforms, multi-region orchestration, or exposing Kubernetes-style APIs to external customers. That exposure model makes its authorization logic security-critical.
Virtual Workspaces and Cross-Tenant Data Access
API providers expose schemas through an API export. Consumers bind to them via an API binding, which includes permission claims for accessing pre-existing resources like secrets. A virtual workspace gives providers a filtered view of consumer data by backing its storage with a label-selector informer against the main KCP server. A controller watches all API bindings and labels matching objects, so only those objects appear in the virtual workspace storage layer. The request pipeline runs authentication, authorization, and admission before the handler serves results.
A Feature Extension That Exposed the Missing Authorization Check
The team was adding verb-level granularity to permission claims. Previously, consumers granted full or no access to a resource type. Before extending the authorizer, Beckers checked the documentation. It said the authorizer verified that the consumer had bound to the provider’s API through an API binding. The code did not match the docs. The authorizer only checked the API export on the provider side. The consumer-side binding check had never been implemented.
From CVSS 4.5 to 9.6 via ClusterRoleBindings
The initial CVSS score was 4.5. List and get operations were unaffected because the label-selector mechanism still filtered results at the storage layer. Create and delete operations bypassed the missing check. An attacker needed permissions to create an API export and a target workspace’s logical cluster ID, a 15-character random string. Then came the key question: do ClusterRoles and ClusterRoleBindings count as built-in resources? They do. Creating a cluster-admin ClusterRoleBinding in a target workspace gave full access. The score moved to near 10.
Fix and Coordinated Disclosure
The fix added a new authorizer to the dynamic virtual workspace’s authorizer chain. It checked consumer-side API bindings and validated the requested verb. For coordinated disclosure, the team used GitHub’s security advisory feature: a private report and a private fork for patch development, with notifications to the mailing list and Slack that included no technical details. GitHub acted as a CNA and assigned the CVE ID directly from the report. The team pre-announced the upcoming patch so users could schedule time to apply it.
Q&A
How hard is it to guess a logical cluster ID, and how does that affect the 9.6 score? An attacker with read-only permissions on the workspace list could extract the ID, so the possibility exists regardless of whether the string is guessable. ▶ 30:09
Could the virtual workspace URL be used to enumerate workspaces? No. The wildcard list endpoint was still constrained by the label selector, so workspaces with no matching API binding label were invisible in listing results. ▶ 31:14
Why async labeling rather than on-the-fly authorization checks per request? The virtual workspace storage uses an informer-based cache against KCP. Checking membership at request time would require listing all objects across all workspaces, consuming too much memory per request. ▶ 32:44
Notable Quotes
problem is this check didn’t exist. So Marvin Beckers · ▶ 13:55
It was surely not on a Friday, Marvin Beckers · ▶ 23:02
uh yeah all all code has vulnerabilities Marvin Beckers · ▶ 24:19
Key Takeaways
- A missing consumer-side binding check let any API export holder create or delete objects in arbitrary workspaces.
- ClusterRoles and ClusterRoleBindings are built-in resources, turning a moderate CVSS 4.5 bug into a near-10 privilege escalation.
- GitHub security advisories handle private triage, private forks, and CVE assignment in one place, reducing coordination overhead.
About the Speaker(s)
Marvin Beckers is a Cloud Software Engineer at ClickHouse and a maintainer for the KCP project, as well as a CNCF Ambassador. He began his career as a Linux sysadmin and shifted into software engineering while automating Kubernetes cluster operations.