What SIG Storage Owns and Maintains

▶ Watch (1:00)

SIG Storage maintains the Kubernetes APIs for persistent volumes, persistent volume claims, storage classes, and volume snapshots. It also owns the Container Storage Interface (CSI) spec and roughly eight CSI sidecar repositories. Sidecars watch Kubernetes API objects and trigger operations against a CSI driver. The group also designed the Container Object Storage Interface (COSI), which lets users provision object buckets the same way PVCs provision block or file volumes.

Kubernetes 1.35: Four Storage Changes

▶ Watch (2:28)

Three features shipped in 1.35. Service account tokens for the CSI secrets field moved directly to beta, skipping alpha because of security concerns with passing sensitive tokens through the non-dedicated volume context field. Mutable volume attach limits, which prevent permanent scheduling failures when GPUs or network interfaces consume attachment slots after CSI initializes, moved to beta-on-by-default. A new alpha feature makes PV node affinity mutable so that volume migrations across zones do not leave the scheduler working from stale topology data. A second alpha feature teaches the cluster autoscaler about CSI attach limits so it spins up the right number of nodes for pending pods.

Kubernetes 1.36: Volume Group Snapshots Reach GA

▶ Watch (9:40)

Volume group snapshots graduate to GA in 1.36. The feature lets users or backup software select multiple PVCs by label and take a single crash-consistent snapshot of all of them. The API mirrors the existing volume snapshot API. Objects use the same pattern: a namespaced VolumeGroupSnapshot object selects PVCs via label selector, and the snapshot controller creates individual VolumeSnapshot objects for each PVC once the group snapshot is ready. The v1 API is identical to v1beta2, so no conversion webhook is needed.

SELinux Mount Options: Breaking 2% of Clusters Safely

▶ Watch (11:58)

Applying SELinux labels by scanning every file on a volume at pod start can take minutes or even hours on volumes with billions of files, and it happens every start. A mount option can apply the label instantly, but it breaks one pattern: a privileged and unprivileged pod sharing the same volume simultaneously. Red Hat telemetry shows only about 2% of clusters do this. In 1.36 all feature gates are enabled but nothing breaks yet. Metrics and events flag what would break on upgrade. Affected clusters can opt out via seLinuxChangePolicy in the pod security context and upgrade safely to 1.37.

Changed Block Tracking and the Git Repo Plugin Removal

▶ Watch (15:19)

Changed block tracking reaches beta in 1.36. The feature exposes which storage blocks changed between two snapshots, the primary use case being incremental backup. Because a large volume can have billions of changed blocks, the data streams over a new gRPC service exposed by the external-snapshot-metadata sidecar rather than through the API server. The sidecar handles authentication, authorization, and encryption. Separately, the git repo volume plugin is disabled in 1.36 and cannot be re-enabled. It predates init containers and was the target of multiple CVEs, most recently in 2024. Init containers or the git-sync sidecar replace it.

Design Work and How to Get Involved

▶ Watch (19:57)

Two designs are still in progress. SIG Storage and SIG Apps are debating how to expand volumes in StatefulSets by editing the VolumeClaimTemplate and propagating changes to all PVCs automatically. The group also wants to merge all eight CSI sidecar repositories into one, reduce maintenance overhead, and eventually produce a single binary to save CPU and memory at runtime. Volume health remains alpha. SIG Storage meets bi-weekly on Thursdays at 18:00 European time. CSI has two calls per week. The SIG landing page lists all Slack channels and sub-project meeting links.

Notable Quotes

I have seen minutes I have seen even hours Jan Šafránek · ▶ 12:37

uh my employ employer as a uh Kubernetes vendor we uh see in our telemetry that only about 2% of the clusters would be affected 90 98% of the clusters will be just fine Jan Šafránek · ▶ 13:43

this uh volume plug-in was target of few CVEes Jan Šafránek · ▶ 17:44

reacting to metrics is pretty hard Jan Šafránek · ▶ 21:23

Key Takeaways

  • Volume group snapshots are GA in 1.36, using label selectors to snapshot multiple PVCs crash-consistently.
  • SELinux mount option support breaks 2% of clusters; 1.36 gives them metrics and an opt-out before 1.37 enforces the change.
  • The git repo volume plugin is disabled in 1.36 after multiple CVEs; use init containers or git-sync instead.

About the Speakers

Xing Yang is a Tech Lead in the Cloud Native Storage team at VMware by Broadcom, co-chair of CNCF TAG Infrastructure, co-chair of Kubernetes SIG Storage, co-chair of the Data Protection WG, and a maintainer in Kubernetes CSI.

Jan Šafránek is a Senior Principal Software Engineer at Red Hat. He has worked on Kubernetes storage for more than eight years and is one of the founding members of SIG Storage. He authored the PersistentVolume controller, dynamic provisioning, and StorageClass.