{"resourceId":"nvidia-cuda-13-4-memory-enforced-gpu-sharing-kubernetes-20261027","versions":[{"version":"external-746933e6694b1700656200a0ec34642fb8dda1c78962dda27581be376307d648","resource":{"id":"nvidia-cuda-13-4-memory-enforced-gpu-sharing-kubernetes-20261027","title":"NVIDIA documents CUDA 13.4 memory-enforced fractional GPU sharing for Kubernetes, with an explicit shared-error-domain trade-off","organization":"NVIDIA","sector":"Research computing / shared GPU clusters (vendor platform)","geography":"Global","publishedAt":"2026-10-27","publicationDate":"2026-10-27","eventDate":null,"sourceName":"NVIDIA Technical Blog","sourceLabel":"NVIDIA Technical Blog: Memory-Enforced GPU Sharing in Kubernetes with NVIDIA CUDA 13.4","sourceUrl":"https://developer.nvidia.com/blog/memory-enforced-gpu-sharing-in-kubernetes-with-nvidia-cuda-13-4/","evidenceClass":"vendor-claim","outcomeClass":"emerging","topics":["infrastructure","operating-model","data-security"],"finding":"NVIDIA describes CUDA 13.4 GPU-memory enforcement for fractional GPU sharing in Kubernetes via the GPU Fractioning Operator and the KAI Scheduler (v0.18.0 NvFractions mode). It offers two modes: time-slicing, where each workload keeps its own error domain, and SM-sharing, where co-located workloads share one error domain and should be mutually trusted.","sledRelevance":"University research computing centers, state HPC consortia and campus AI teaching labs routinely oversubscribe scarce GPUs across many users; hard per-workload memory limits make fractional allocation more predictable, but the error-domain choice determines whether one tenant's fault can affect another.","evidence":"Vendor technical post describing the feature, modes and configuration; no measured utilization, throughput or isolation results are reported in the inspected text.","architectureImplications":"Adds a scheduler-plus-operator layer (KAI Scheduler NvFractions, GPU Fractioning Operator) on top of the GPU Operator and driver stack; clusters must pick time-slicing or SM-sharing per pool.","governanceImplications":"Shared-cluster policies need a rule for which tenants may share an SM-sharing pool (for example, same lab or same project) versus time-sliced pools for unrelated users.","securityPrivacyImplications":"SM-sharing puts co-located workloads in one error domain; NVIDIA advises it only for workloads trusted to run together, so it should not mix different research groups or regulated data projects.","caveats":"Vendor post without measured results or independent testing; feature depends on a newer driver branch and specific GPU Operator versions (see the project README).","streamIds":["nvidia"],"roles":{"sales":"Pitch this to university research computing directors and state HPC consortium leads who face GPU queues that are always longer than supply. The opening is a utilization and fairness review: which jobs could share GPUs, and under what tenancy rules. Do not promise utilization gains; NVIDIA's post reports none. Position the conversation around policy as much as technology, because SM-sharing requires trusted co-tenants. A small paid assessment that maps workloads to time-sliced versus SM-sharing pools is a credible first step before any platform upgrade commitment.","engineering":"Engineering should treat this as a stack upgrade, not a toggle: KAI Scheduler v0.18.0 NvFractions, the GPU Fractioning Operator, a supported GPU Operator release and a newer driver branch. Validate that per-workload memory limits actually terminate offenders as documented, and test both modes. Time-slicing preserves separate error domains, while SM-sharing trades isolation for concurrency. Measure throughput, queue wait and fault behavior on representative campus jobs before production. Keep regulated or multi-group workloads out of SM-sharing pools and record the configuration as part of the cluster's security baseline.","delivery":"Delivery should plan a staged pilot on one non-production GPU node pool, with a change window for the driver and operator upgrades. Define tenancy tiers with the research computing governance committee before enabling SM-sharing. Publish user guidance on fraction requests and on what an out-of-memory termination means. Acceptance should rest on measured queue-wait and utilization changes against a baseline month, plus a documented fault-injection test. Monitor for silent telemetry failure, which the project README warns about, and retain a rollback to whole-GPU allocation."},"roleDetails":{"sales":{"customerProblem":"GPU demand on shared campus and consortium clusters exceeds supply, and whole-GPU allocation strands memory and compute.","stakeholders":["Research computing director","HPC operations lead","CISO or research security officer","Faculty governance committee"],"discoveryQuestions":["Which workloads use less than a full GPU today?","Which groups can safely share a fault domain?","What baseline queue-wait and utilization data exists?"],"valueHypothesis":"Enforced memory fractions could let more small jobs run per GPU with predictable limits, if tenancy rules are set first.","potentialEngagement":"Fixed-scope GPU sharing assessment and one-pool pilot plan.","unsupportedClaims":["No measured utilization or throughput gain is reported.","No independent test of isolation is cited."]},"engineering":{"fit":"Kubernetes-based GPU clusters with many small or interactive jobs.","architectureIntegration":"KAI Scheduler NvFractions mode plus GPU Fractioning Operator layered on GPU Operator and driver stack.","prerequisites":["Kubernetes cluster with GPU Operator","Upgraded driver branch and container runtime support","KAI Scheduler v0.18.0"],"constraints":["SM-sharing places workloads in one error domain","Vendor post reports no measured results"],"security":"Use time-slicing for untrusted or multi-group tenants; restrict SM-sharing to trusted co-tenants.","proofOfValue":["Baseline vs pilot queue wait and GPU utilization","Fault-injection test in each mode","Verified OOM enforcement at the configured limit"]},"delivery":{"work":"Upgrade one node pool, configure fractions, define tenancy tiers and run a measured pilot.","dependencies":["Driver and operator upgrade window","Research governance approval of tenancy tiers"],"ownership":"Research computing operations with security office sign-off.","skillsAdoption":"Train platform staff on scheduler/operator; publish researcher guidance.","governanceCheckpoints":["Tenancy policy approved before SM-sharing","Security review of shared pools"],"acceptanceCriteria":["Measured improvement vs baseline month","No cross-tenant impact in time-sliced pool tests"],"risks":["Silent telemetry failure","Cross-tenant fault impact in SM-sharing","Upgrade regressions"]}},"retrievedAt":"2026-10-28T03:03:12Z","enrichedAt":"2026-10-28T03:12:33Z","enrichmentBasis":"retrieved source","accessibilityWorkforceImplications":"Platform teams need Kubernetes, GPU Operator and scheduler skills; researchers need guidance on requesting fractions and understanding OOM termination.","procurementImplications":"No licensing change is stated; adoption is an upgrade and configuration effort rather than a purchase, but may require driver-branch and OS/runtime upgrades.","operatingModelImplications":"Research computing operations would need tenancy tiers (trusted SM-sharing pools vs isolated time-sliced pools) and chargeback per fraction.","sourceVerification":{"openedUrl":"https://developer.nvidia.com/blog/memory-enforced-gpu-sharing-in-kubernetes-with-nvidia-cuda-13-4/","referenceExcerpt":"SM-sharing puts the workloads in one error domain, so use it for workloads you trust to run together.","promptVersion":"sled-research-v4.0","model":null,"basis":"agent-reported inspection"}}}]}