Strategic PartnersIssue 119 records
NVIDIA
This edition shifts from yesterday's NPS/NIM-profile archive set to newly archived platform and scrutiny evidence: September 15 AI Infra Summit claims for Vera Rubin/DSX MaxLPS tokens-per-megawatt, the DSX Facilities v2.0 250 MW-class reference (198–330 kW cabinets, 45C liquid cooling), Dynamo shadow engine recovery (7.3s vs 283s) gated on Kubernetes DRA, CoreWeave's first cloud GB300 NVL72 with Dell/Vertiv, NIM Operator 3.x and GPU Operator 26.7.0 platform constraints, Military.com's independent NPS ~135 kW/storage narrative, and a clear contradiction between NVIDIA's ~1–8% CC overhead tables and Tinfoil's independent finding that CC-naive engines can see ~30% to >100% ITL penalties.
- Evidence records
- 9
- Cross-source patterns
- 3
- Evidence classes
- 4 vendor claim3 standards or public-body guidance1 independent research1 independent reporting
- Outcomes
- 5 cautionary3 emerging1 mixed
- Source freshness
- 4 recent3 new this fortnight2 undated
- Research completed
- 2026-09-17
Choose a role to see its takeaway beside every record in the ledger.
Patterns across the evidence
Synthesis · Lighthouse Advisory interpretation
Confidential Computing overhead is engine- and path-scoped
NVIDIA's HGX B300 SGLang tables show roughly 1–8% steady-state inference impact, while Tinfoil's independent Blackwell study shows ~1% on CC-aware engines but ~30% to >100% ITL penalties on CC-naive stacks—plus training losses of 18–32%—driven by bounce-buffer and sync taxes rather than GPU math.
Operating questionIf CC is mandatory, which engine build and concurrency regime were measured under CC-on for this workload card?
Supporting evidenceNVIDIA claims Confidential Computing inference overhead roughly 1–8% on HGX B300Tinfoil
AI-factory power codesign does not transfer 1:1 to SLED rooms
DSX Facilities v2.0 documents 198–330 kW cabinets and 250 MW-class campuses, while the Sept 15 AI Infra Summit recap highlights MaxLPS reclaim and Vera Rubin tokens-per-megawatt gains in partner factories—useful directionally but cautionary for legacy education/government rooms.
Operating questionWhat measured rack kW, loop temperature, and utility ceiling exist before citing AI-factory density claims?
Supporting evidenceDSX Facilities Infrastructure Reference Design v2.0 for 250 MW-class AI factoriesAI Infra Summit: Vera Rubin, DSX MaxLPS, and partner tokens-per-watt resultsMilitary.com
Modern serving resilience features gate on Kubernetes DRA floors
Dynamo shadow engine recovery and GPU Operator GPUCluster DRA both require Kubernetes 1.34.x-class DRA, and NIM Operator 3.x adds DRA/multi-node Ray paths—so regulated clusters stuck on older distros cannot claim seconds-scale failover or DRA allocation without a platform upgrade.
Operating questionIs the estate on Kubernetes 1.34.2+ with DRA enabled before promising Dynamo shadow recovery or GPUCluster allocation?
Supporting evidenceDynamo shadow engine recovery: 7.3s vs 283s cold restart on B200GPU Operator 26.7.0: GPUCluster DRA management and passthrough allocation pitfallsNIM Operator 3.x: multi-node Ray, DRA, gov-ready, and known caching constraints
Evidence ledger
Full record · every source keeps its link and limitations
AI Infra Summit: Vera Rubin, DSX MaxLPS, and partner tokens-per-watt results
On September 15, 2026 NVIDIA published an AI Infra Summit recap claiming Vera Rubin NVL72 MLPerf Inference v6.1 preview throughput up to 3.7x vs GB300, Lambda DSX MaxLPS validation (+24% token throughput / +23% perf-per-watt within a fixed power budget), Emerald AI flexible-load demos with Silicon Valley Power, and SemiAnalysis AgentX claims of up to 30x throughput per megawatt vs GB300 for agentic coding sessions.
Why it matters, evidence and limitations
- Why it matters
- Relevant as ecosystem direction on power-constrained AI factories and agentic inference economics. Most claims are hyperscale/cloud-partner measured; SLED campuses rarely match 250 MW-class DSX sites. Treat Lambda/Emerald results as transferability signals for power budgeting—not turnkey campus outcomes.
- Evidence and measured results
- NVIDIA Writers post dated September 15, 2026 reports Ian Buck AI Infra Summit remarks; lists Amazon Annapurna NVHBM and d-Matrix NVLink Fusion collaborations; states Lambda ran 19 nodes in a 16-node power budget (+24% tokens/s, +23% perf/watt) and cites Vera Rubin MLPerf preview up to 3.7x GB300 and AgentX up to 30x tokens/MW vs GB300.
- Limitations and uncertainty
- Vendor-authored summit recap aggregating partner demos and MLPerf preview numbers without independent replication in this source. Hyperscale power envelopes and agentic coding AgentX trajectories may not map to campus/agency workloads.
DSX Facilities Infrastructure Reference Design v2.0 for 250 MW-class AI factories
NVIDIA's DSX Facilities Infrastructure Reference Design (Design Guide v2.0, August 19, 2026) documents a 250 MW-class example site—157 acres, 1,536 GPU racks, 110,592 GPUs—with 34.5 kV campus MV backbone, 45°C liquid-cooling design point with dry coolers (no evaporative water), N+1 CDUs, cabinet TDP from 198 kW (MGX Gen 1.1) to 330 kW (Vera Rubin NVL72), and cooling plant not UPS-backed on the GPU side.
Why it matters, evidence and limitations
- Why it matters
- Sets the true facility bar for NVIDIA AI-factory scale. Directly cautionary for SLED: most campuses and agencies cannot host 198–330 kW cabinets or 250 MW campuses. Use as a sizing/transferability boundary when partners pitch AI-factory language for education or government rooms.
- Evidence and measured results
- Docs page cites NVIDIA DSX Facilities Infrastructure Design Guide v2.0 (August 19, 2026); example table lists 250 MW-class IT load, 96 Scalable Units, 157-acre footprint, 240 MW compute IT+mechanical and 22.8 MW core loads; text states 45°C liquid-cooling design point, dry coolers with no evaporative water use, chillers on 4.16 kV, and cabinet TDP 198–330 kW.
- Limitations and uncertainty
- Reference design for NVIDIA Cloud Partners at hyperscale; not a measured SLED deployment. Supplemental figures via NVOnline. Actual sites vary.
Dynamo shadow engine recovery: 7.3s vs 283s cold restart on B200
NVIDIA documents Dynamo shadow engine recovery (preview): a preinitialized standby engine shares weights via GPU Memory Service and resumes serving in 7.3 seconds after a worker SIGKILL versus 283 seconds cold restart in a two-worker GLM-5.2 NVFP4 deployment on B200 (TP=8), with large TTFT/decode improvements. Requires Kubernetes 1.34+ DRA and NVIDIA GPU DRA driver; vLLM primary backend; does not cover hardware/node/multi-node failures; promoted shadows start with empty KV cache.
Why it matters, evidence and limitations
- Why it matters
- Matters for agencies/universities running multi-tenant LLM APIs where process crashes cause SLA breaches. Value is real only if the estate can run K8s 1.34+ DRA—often a blocker for regulated clusters still on older distributions.
- Evidence and measured results
- Blog dated Aug 25, 2026 reports deliberate worker kill experiment: cold restart 283s vs shadow 7.3s (~39x); TTFT p50 after fault 23,815 ms vs 1,311 ms; decode 12 vs 46 tok/s/user; lists DRA/K8s 1.34+, vLLM primary, empty KV after promotion, and no coverage of hardware/node failures.
- Limitations and uncertainty
- Vendor-measured synthetic load (0.7 rps, 32k/1k tokens); preview limitations; primary backend vLLM only; empty KV causes post-cutover TTFT bump.
NVIDIA claims Confidential Computing inference overhead roughly 1–8% on HGX B300
NVIDIA reports Blackwell Confidential Computing (RTX PRO 6000, HGX B200/B300) with NRAS attestation and NVLink encryption on multi-GPU HGX, stating inference performance up to 98% of non-CC. On HGX B300 with Qwen 3.5 397B-A17B FP8 in SGLang, tabulated delta vs CC-off ranges about -1% to -8% for throughput/TPOT across concurrencies 4–256. Optimizations cited: FlashInfer CC-safe autotuner, SGLang async D2H worker, piecewise CUDA graphs.
Why it matters, evidence and limitations
- Why it matters
- Attractive for regulated SLED inference needing in-use protection. Claims are vendor-measured on a CC-aware SGLang stack—must be cross-checked against independent results showing much higher overhead for CC-naive engines.
- Evidence and measured results
- Blog dated July 2, 2026 provides Table 1 relative performance for Qwen3.5-397B-A17B-FP8 on HGX B300/Intel TDX; software stack lists driver 595.71.05, CUDA 13.2, SGLang v0.5.12-cu130; states attestation is typically one-time at startup.
- Limitations and uncertainty
- Single vendor model/engine/config; steady-state inference focus; does not report training or CC-naive engine penalties highlighted elsewhere.
Independent Tinfoil benchmarks: Blackwell CC overhead from ~1% to >100% by engine
Tinfoil's public Blackwell/Hopper CC study (Intel TDX + NVIDIA CC) finds GPU math near-free but bounce-buffer and sync taxes dominate: ITL overhead ~1% with CC-aware TRT-LLM/SGLang to ~30%+ with CC-naive vLLM in compute-bound regimes and >100% at low concurrency; training throughput loss 18–32% on NeMo AutoModel; pinned H2D collapses to ~0.20x under CC while pageable keeps ~0.78x; kernel launch 3.5x slower on B300; Blackwell multi-GPU loses multicast/NVLS.
Why it matters, evidence and limitations
- Why it matters
- Primary independent counterweight to vendor won't-slow-you-down messaging. For SLED regulated inference, engine choice and CUDA-graph discipline become security-performance controls—not optional tuning.
- Evidence and measured results
- Long-form post Jun 23, 2026 details 8xB300 and 8xH200 TDX setups, engine versions, and tables for LoRA vs full SFT training losses (18–32%), ITL overhead by concurrency, and microbenchmarks (kernel launch, pinned vs pageable bandwidth).
- Limitations and uncertainty
- Vendor-of-CC-services (Tinfoil) with incentive to publish; still the clearest public multi-engine Blackwell CC map. Some root causes marked as under discussion with NVIDIA.
NIM Operator 3.x: multi-node Ray, DRA, gov-ready, and known caching constraints
NIM Operator 3.x release notes add multi-node NIM via Ray (v3.1.1), Model-Free mode, Gateway API routing, DRA GPU allocation (v1 API), experimental Kata/CC sandbox and Dynamo CRDs, and government-ready designation (v3.0.1). Known issues include Too many open files on 49B+ caches (raise containerd LimitNOFILE), NeMo Retriever tensorrt filter failures, and deprecation of NeMo Microservices CRDs toward NeMo Platform. ConfigMap size fixes appear in 3.1.2.
Why it matters, evidence and limitations
- Why it matters
- Directly actionable for SLED Kubernetes NIM estates—especially regulated (gov-ready) and multi-node large models. Deprecations and file-limit/cache pitfalls are delivery blockers if ignored.
- Evidence and measured results
- Opened latest release notes page documenting v3.1.2–v3.0.0 features and Known Issues sections for large-model open-file limits, Retriever tensorrt filtering, HF permission warnings, and gov-ready note in v3.0.1.
- Limitations and uncertainty
- Living docs without a single publicationDate; known issues evolve by version. Air-gap and OpenShift RAG sample limits called out in older notes.
CoreWeave announces first cloud GB300 NVL72 bring-up with Dell racks
CoreWeave reports industry-first cloud bring-up of NVIDIA GB300 NVL72 in Dell integrated rack-scale systems with Switch and Vertiv, citing up to 10x user responsiveness and 5x throughput-per-watt vs Hopper (and derivative 50x reasoning output claim), 72 Blackwell Ultra GPUs / 36 Grace CPUs / 18 BlueField-3 DPUs, expanded HBM3e per rack, and Quantum-X800 800 Gb/s networking. Schema datePublished 2026-09-08.
Why it matters, evidence and limitations
- Why it matters
- Shows commercial cloud availability path for GB300 without on-prem facility conversion. Limited direct SLED measurement—useful for hybrid/sovereign-cloud options and for understanding partner stack (Dell/Vertiv/Switch) dependencies.
- Evidence and measured results
- Opened CoreWeave blog with JSON-LD datePublished 2026-09-08; describes Dell rack-scale housing, partner list Dell/Switch/Vertiv, and GB300 hardware composition and marketing performance multipliers vs Hopper.
- Limitations and uncertainty
- Marketing multipliers vs Hopper without workload cards in-post; customer quote from Moonvalley (media), not government. Broader availability still pending.
Military.com independent report: NPS DGX GB300 online with ~135 kW and data-path bottlenecks
Military.com's Haley Fuller reports NPS brought the first U.S. military DGX GB300 online for education/research, stressing storage/network bottlenecks (DDN interview), partner stack (NVIDIA donation, Vertiv power/cooling, VAST, DDN), ~135 kW peak with liquid cooling, and the gap between campus research systems and ship/expeditionary power-space limits before fleet tooling.
Why it matters, evidence and limitations
- Why it matters
- Newly relevant independent narrative beyond archived NPS press: reinforces facility power figures and adds data-integrity/poisoning and fleet-transfer constraints useful for education and government buyers evaluating Blackwell on-prem.
- Evidence and measured results
- Article published Sep 6, 2026 quotes DDN's Kevin Delane on data-path bottlenecks; states 72 Blackwell Ultra + 36 Grace; lists Vertiv/DDN/VAST/NVIDIA roles; explicitly notes ~135 kW peak and liquid cooling; discusses training-data integrity and DoD continuous monitoring expectations.
- Limitations and uncertainty
- Relies partly on vendor executive interview (DDN); not a controlled benchmark. NPS commissioning already archived from primary sources—this adds independent framing and fleet-transfer caution.
GPU Operator 26.7.0: GPUCluster DRA management and passthrough allocation pitfalls
GPU Operator 26.7.0 adds GPUCluster CR to manage DRA Driver for GPUs (full GPU and preconfigured MIG via ResourceClaims), KubeVirt VFIO passthrough with DRA, driver 595.91.07 stack updates, and in-place patch restarts. Requires Kubernetes v1.34.2+, driver 580+, CDI runtime; cannot mix GPUCluster with ClusterPolicy. Known issue: concurrent container and VFIO claims on the same physical GPU can race—submit serially or dedicate nodes.
Why it matters, evidence and limitations
- Why it matters
- Critical platform constraint for SLED clusters adopting Dynamo shadow recovery or NIM DRA allocation—both need modern K8s DRA. Mixing VM passthrough and containers without node dedication is a concrete failure mode.
- Evidence and measured results
- Opened latest release notes for 26.7.0 New Features and Known Issues describing GPUCluster, K8s v1.34.2+ requirement, mutual exclusion with ClusterPolicy, and DRA passthrough double-allocation race.
- Limitations and uncertainty
- Release notes are version living docs; some DRA capabilities remain alpha/disabled by default.
How to read this edition
Source findings, measured results and limitations come from the cited publications. Patterns, operating questions, role takeaways and implementation considerations are Lighthouse Advisory interpretation, stated as questions to validate locally rather than guaranteed outcomes. Vendor and operator claims are labeled as claims. Full research method.
- Vendor claim
- A supplier-provided assertion that has not been upgraded to independent evidence.
- Standards or public-body guidance
- Normative or advisory guidance from a standards body or public institution.
- Independent research
- Research conducted outside the implementing organization.
- Independent reporting
- Independent reporting with attributable sources but without a formal evaluation design.