AI
The Telecom Constraint That Breaks “Build Once, Run Anywhere”
Telecom infrastructure must deliver five-nines availability, meaning less than six minutes of downtime per year. Governments in multiple countries enforce that SLA with financial penalties. A rollout window is typically four hours, from 2 a.m. to 6 a.m. Any issue during that window requires a full rollback. There is no time to debug. Every 5G antenna runs a different SOC, and academic HPC clusters mix chips spanning a decade of procurement cycles. The same AMD64 container can silently break on hardware from the same year if the underlying node differs in even one kernel flag.
SYLVA Validation: Nine Platforms, 22 Vendors, One Compatibility Gap
SYLVA is a Linux Foundation project funded by five European telecom operators and two vendors. It ships a Kubernetes software stack tailored for telecom and edge workloads. The validation center currently covers nine validated platforms and 22 vendor products. The missing piece was proving that a vendor application built on one hardware stack would actually run correctly on a third-party Kubernetes platform with different underlying hardware and OS. Image compatibility tooling is now integrated into the SYLVA validation center, using Kubernetes jobs to automate that check before any production rollout.
NFD Artifacts: Storing Build-Environment Metadata Inside the Image
When SYLVA approached the Kubernetes community, the core idea was straightforward. NFD already scrapes detailed node information, from CPU flags and PCI devices down to whether a USB device was plugged in at scrape time. The new work packages that information as an ORAS artifact stored alongside the container image in the registry. ORAS extends the OCI registry standard to hold full artifacts, not just images. Each image now carries a YAML describing the exact node where it was built. DockerHub already supports the ORAS API, so no custom registry infrastructure is required.
Node Feature Groups and the NFD Scheduler Plugin
NFD gained two additions for this work. First, a CLI that can query cluster nodes and compare them against an image artifact. Second, a Node Feature Group CRD that lets operators catalog infrastructure into named groups, such as GPU training nodes, high-performance compute nodes, and general-purpose nodes. A new scheduler plugin, currently an open pull request targeting the next NFD release, reads both the node labels and the image artifact. Because both use the same NFD API schema, matching is direct. If no node matches the artifact, the pod is not scheduled at all.
Live Demo: Routing and Rejecting Pods by Hardware Compatibility
The demo ran a two-node kind cluster with NFD deployed and the Node Feature Group API enabled via feature gate. Eduardo pushed an image plus its NFD-generated ORAS artifact to the local kind registry, then deployed the scheduler configured to use the NFD plugin. The scheduler matched artifact metadata against node labels and placed the pod on the correct node. The demo also showed the rejection path: a pod targeting an incompatible node receives a policy failure and does not schedule. Building the CLI and aligning it with NFD’s schema across multiple vendor stakeholders took close to a full year.
Q&A
How does the system handle post-deployment discovery that two nodes assumed to be identical are actually different? The NFD matcher engine supports ranges of values, not just exact matches, so operators can encode backward and forward compatibility tolerances in Node Feature Group rules rather than treating a single missing flag as a hard block. ▶ 29:12
Does this work with multi-architecture container builds the way Docker buildx does? Every container image gets its own ORAS artifact recording where it was built, so multiple architecture-specific images pushed to the same registry each carry separate metadata, and the NFD scheduler plugin selects the correct one per node. ▶ 33:20
Notable Quotes
container can one doesn’t mean it work in telecom industry because it has to meet uh the SOA and the latency and other requirements Chaoyi Huang · ▶ 07:48
the low out the loout is done in so short time window there is no chance to de debug any issue. any issue happened have to lower back have to to lower back. Chaoyi Huang · ▶ 08:30
NFD discovers everything so that’s not the problem the problem that you’re saying of understanding backwards and forwards compatibility it can be man uh managed via the matchers right Eduardo Arango Gutierez · ▶ 30:52
it looks very simple but actually took us almost like a full year to get here uh because all the different permutations the problems that we were facing Eduardo Arango Gutierez · ▶ 25:17
Key Takeaways
- SYLVA’s four-hour rollout window makes pre-deployment node compatibility checks mandatory, not optional.
- NFD now embeds full node metadata into ORAS artifacts stored alongside container images in the registry.
- The NFD scheduler plugin rejects pods outright when no cluster node matches the image’s build-environment artifact.
About the Speaker(s)
Eduardo Arango Gutierrez is a Senior Systems Software Engineer at NVIDIA on the Cloud Native Technologies team, focused on building and deploying containers in distributed environments.
Chaoyi Huang (Joe Huang) is an expert at Huawei Technology Co. with around ten years in the cloud industry. He created memory overcommit technology and designed algorithms to detect performance degradation in co-located virtual machines, and is an active contributor to Kubernetes, CNCF, and SYLVA.