AI
Why a Working GPU Cluster Is Not a Shareable Platform
Getting GPUs to run is not the hard part. Sharing them safely across teams is. The production deployment behind this talk ran H200 nodes with more than 137 GB of GPU memory each, up to eight cards per Dell AI server, in a fully air-gapped environment. Installing the platform alone required downloading more than 2 TB of dependencies using local mirrors. The real operational risks were noisy-neighbor interference, queue starvation, mismatched MIG behavior, and driver drift that blocked cluster-wide visibility.
Three Lenses Every Platform Decision Must Pass
Every architectural choice was evaluated against three questions: safe (one team cannot break another), fair (contention is deterministic, not political), and efficient (optimized for useful outcomes per GPU hour, not utilization graphs). Three personas drove the requirements. The end user needs fast access and stable latency. The LLM ops team needs repeatable deployments and observability. The tenant admin needs cost visibility and hard boundaries. If any of those personas loses, platform adoption collapses.
GitOps as the Enforcement Mechanism
Argo CD and Kustomize became the single source of truth for infrastructure, operations, and tenant configuration. Each tenant was provisioned from a standard Kustomize build that bundled namespace configuration, RBAC, network policy, quotas, and supporting networking. A Git pull request became the tenant provisioning workflow. No tickets, no manual steps, no tribal knowledge. For an avionics customer in a highly regulated environment, the auditability and rollback capability mattered more than the automation itself.
Hardware Surprises on Bare Metal: RDMA, NICs, and Driver Coupling
Bare metal forces attention to the actual hardware mix. Nvidia ConnectX-6 and ConnectX-7 cards were managed by the Nvidia network operator for RDMA and GPU Direct flows, while other NIC paths required separate handling across dedicated VLANs for NFS, backup, and the main machine network. An RDMA collision at kernel level between the Nvidia Mellanox ConnectX card and the Intel NIC blocked SR-IOV entirely. The fix was blacklisting the Intel driver. Moving to open kernel modules and DMA buffers then reduced upgrade fragility by decoupling drivers from kernel dependencies.
Fairness Through NVIDIA KAI and Scheduling Constraints
Without hard rules, the loudest team wins. Berton used NVIDIA KAI to enforce hard per-tenant GPU caps, explicit preemption posture, and priority classes. Interactive workloads get the highest scheduling priority because users need fast responses. Training jobs run in batch overnight. MIG slicing, time slicing, and full GPU allocation are chosen per workload type, not by ideology. Canary pools run for 48 hours before each upgrade, and any component that misbehaves rolls back via Argo CD to a known-stable Git state.
Observability and the Three-Persona Demo
Per-tenant monitoring included chargeback views, scheduler reports, and GPU-per-hour visibility in Grafana using NVIDIA DCGM dashboards showing power usage, temperature, SM clock speed, and per-MIG-profile memory. The demo walked three personas. William, the LLM ops engineer, deployed a Microsoft Phi Mini model from Hugging Face through OpenShift AI using vLLM on an H200. Michael, the OpenShift admin, monitored real-time hardware metrics across all deployments. Mark, the Python developer, connected Visual Studio Code to the internal API endpoint using the open-source Continue AI assistant, with no public cloud dependency.
Notable Quotes
Getting GPUs to run is not the hard part. The hard part is sharing them safely, fairly, and efficiently across teams. Luca Berton · ▶ 00:19
in shared GPU environment, fairness does not happen naturally. Without a clear rule, the loudest teams wins. Luca Berton · ▶ 13:03
The bottom message is mature shared GPU platform are built from gather rays, not heroics. Luca Berton · ▶ 21:56
GitOps was the mechanism that made those principle enforceable by design. Luca Berton · ▶ 22:25
Key Takeaways
- Hard GPU caps via NVIDIA KAI make resource contention predictable and explainable across tenants.
- A kernel-level RDMA collision between Nvidia ConnectX and Intel NICs blocked SR-IOV until the Intel driver was blacklisted.
- GitOps with Argo CD and Kustomize turns a Git pull request into a fully auditable tenant provisioning workflow.
About the Speaker(s)
Luca Berton is a freelance IT consultant at Dell Technologies with over 18 years of experience in AI, cloud technologies, and automation. He educates more than 15,000 students on platforms including Pluralsight and Coursera, and specializes in Kubernetes, Terraform, and AI-ready infrastructure. The lessons in this talk come from a production deployment for an avionics customer in France.