Strategic PartnersIssue 1010 records
NVIDIA
This edition centers on newly archived evidence for NVIDIA AI platforms in education and government contexts: Naval Postgraduate School’s commissioned DGX GB300 (with published ~135 kW peak and liquid-cooling details) and the facility-conversion work required to place Blackwell systems in legacy rooms; University of Florida’s NIM-backed NaviGator campus platform as a vendor-reported hybrid operating model; and engineering constraints that bound production claims—NIM 2.0.12 profile-specific correctness/OOM issues, Operator HPA guidance that rejects CPU metrics, AI Enterprise EOL migration windows, independent scoping of the 2.5x Nemotron Ultra NIM capacity headline, and a Blackwell GPU-CC study showing LLM serving losses driven by a serialized VM–GPU bridge despite near-parity GPU compute.
- Evidence records
- 10
- Cross-source patterns
- 3
- Evidence classes
- 4 standards or public-body guidance3 vendor claim2 independent reporting1 independent research
- Outcomes
- 5 cautionary3 mixed2 emerging
- Source freshness
- 6 undated2 recent1 new this fortnight1 older, newly relevant
- Research completed
- 2026-09-15
Choose a role to see its takeaway beside every record in the ledger.
Patterns across the evidence
Synthesis · Lighthouse Advisory interpretation
Facility conversion gates Blackwell education/government deployments
NPS’s DGX GB300 commissioning and partner facility narrative show that liquid cooling, ~135 kW-class power, and legacy-room conversion are decisive delivery work—not optional afterthoughts to GPU acquisition.
Operating questionBefore promising Blackwell on-prem capacity to a campus or agency, what measured power, water, and liquid-cooling readiness exists in the target room?
Supporting evidenceNaval Postgraduate SchoolNaval Postgraduate School AI Task ForceVertiv
NIM capacity and correctness are profile- and workload-scoped
Independent scoping of the 2.5x Nemotron Ultra NIM claim, NIM 2.0.12 known issues, Operator HPA guidance, and agent load-test experience together show that serving results depend on cache reuse, precision/profile, inference metrics, and configuration—not headline multipliers alone.
Operating questionWhich workload card, NIM profile, and inference metrics will this estate use to accept a capacity or quality claim?
Supporting evidenceMustHave.aiNIM LLM 2.0.12 release notes document profile-specific correctness and OOM constraintsNIM Operator guidance requires inference metrics for HPA and long model-load startup probesNVIDIA production note: AI-Q/LangGraph agent scaling needed NIM CPU fix and phased rollout
Confidential GPU serving still needs bridge-aware runtime engineering
Independent Blackwell GPU-CC measurements show near-parity GPU compute but material LLM serving and KV-restore penalties from a serialized CVM–GPU bridge, while AI Enterprise EOL pressure continues to force model/platform migrations under support and CVE constraints.
Operating questionIf confidential computing is required, what serving-path overhead and attestation gaps are accepted—and which EOS components must be migrated first?
Supporting evidencePhala Network researchers (Hang Yin, Kevin Wang)AI Enterprise EOL notices add immediate and January 2027 NIM migration deadlines
Evidence ledger
Full record · every source keeps its link and limitations
NPS commissions first U.S. military DGX GB300 for defense education and research
NPS publicly commissioned a donated DGX GB300 (72 Blackwell Ultra GPUs / 36 Grace CPUs) under an NPS–NVIDIA CRADA, framing the system as education and applied defense research capacity rather than a finished operational AI product. Leadership emphasized managing demand during ramp-up and measuring value by decision speed, not only compute hours.
Why it matters, evidence and limitations
- Why it matters
- Shows a concrete U.S. government graduate-education path to Blackwell-class on-prem AI: foundation donation, CRADA, and industry partners for storage/power/cooling. SLED buyers should separate commissioning publicity from measured research or instructional outcomes.
- Evidence and measured results
- NPS Public Affairs release dated July 22, 2026 describes ribbon-cutting with NVIDIA CEO and USPACOM leadership; states donation via NPS Foundation; lists Vertiv, DDN, and VAST Data as contributors; quotes CIO on careful demand management as the system comes online.
- Limitations and uncertainty
- Release is commissioning narrative without utilization, latency, model quality, or cost metrics. Partner contributions and NVIDIA donation mean results are not a commercial procurement case study.
NPS documents GB300 peak power (~135 kW) and closed-loop liquid cooling inside existing capacity
NPS publishes operator-facing facts for the donated DGX GB300: 72 Blackwell Ultra GPUs, 36 Grace CPUs, ~135 kW peak compute draw with expected daily operations at 50–80% of peak, and dual closed-loop liquid cooling that rejects heat via roof cooling water. The page states the platform operates within existing power and water capacities.
Why it matters, evidence and limitations
- Why it matters
- Gives SLED facility and procurement teams quantitative planning inputs for Blackwell rack-scale education/research deployments, and clarifies that within-existing-capacity claims still require local site verification.
- Evidence and measured results
- NPS GB300 page lists architecture (18 trays x 4 GPU/2 CPU), FP4 performance, 20 TB HBM, use-case list (weather, cyber, LLM fine-tuning, multi-agent tactics), and Power and Cooling section with 135 kW peak and 50–80% anticipated daily operations.
- Limitations and uncertainty
- Institutional specifications and anticipated utilization bands are not independent measurements. Performance figures (PFLOPS) are product specs, not NPS workload benchmarks.
Vertiv describes NPS room conversion as the gating work for DGX GB300 deployment
Vertiv’s partner narrative states NPS’s existing computer room was not designed for current AI power density, liquid cooling, and thermal load, and that deployment required a three-rack SmartIT module (UPS, high-density PDUs, direct-to-chip cooling, rear-door cooling, CDUs, MGX chassis, seismic anchoring) adapted from NVIDIA’s DGX GB300 reference design.
Why it matters, evidence and limitations
- Why it matters
- Separates GPU acquisition from facility conversion work that many SLED campuses and agencies face when placing Blackwell-class systems in legacy rooms.
- Evidence and measured results
- Article details Liebert APM2 UPS, dual high-density PDUs, DCD47 rear-door units, three XDU CDUs, Open Rack MGX chassis with 1,400A DC busbar, and California seismic anchoring; frames physical readiness, system integration, and deployment confidence as three requirements.
- Limitations and uncertainty
- Vendor marketing of its own module; performance claims cite NVIDIA specifications (>1,000 petaflops) without independent measurement. Complements—not replaces—NPS primary sources.
Independent study: Blackwell GPU-CC LLM losses come from serialized VM-GPU bridge
On Blackwell, GPU-local compute under confidential computing is near parity (BF16 matmul ~0.998x), but LLM serving under Intel TDX + GPU-CC still loses 13–27% throughput; KV-cache restore can more than double. Authors attribute losses to a serialized, high-setup-cost CVM–GPU bridge and show runtime mitigations (e.g., --no-async-scheduling recovers ~57%; worker-thread drain up to ~92% in qualified runs). They also note a fabric-attestation gap for confidential multi-GPU NVSwitch tenants.
Why it matters, evidence and limitations
- Why it matters
- Critical for SLED regulated workloads that assume confidential GPUs erase the performance tax. Agencies should budget serving-path engineering and attestation gaps, not only encrypted VRAM marketing claims.
- Evidence and measured results
- Abstract and introduction report B300 matmul parity, 13–27% serving loss, +131% KV-restore, 34x model-load slowdown with default loader; recovery hierarchy and remaining Fabric Manager/NVSwitch attestation gap are documented in the opened HTML preprint.
- Limitations and uncertainty
- Preprint (not peer-reviewed journal version here). Results are on specific Blackwell platforms and vLLM-centered stacks; mitigations are qualified and may not transfer unchanged.
NIM LLM 2.0.12 release notes document profile-specific correctness and OOM constraints
NIM 2.0.12 adds models and moves to vLLM 0.27.1, but known issues include NVFP4 nemotron-3-nano semantic incorrectness (no workaround—use BF16/FP8), Responses API correctness limits, LoRA rank <=128 on some MoE kernels, KV-cache OOM at default 131k context on several profiles, and GB10 completion timeouts under specific large-request settings.
Why it matters, evidence and limitations
- Why it matters
- SLED engineering teams must treat support-matrix profiles as operational controls: precision, context length, and GPU SKU choices can change correctness and availability, not only speed.
- Evidence and measured results
- Opened release notes list Highlights, Bug Fixes, and Known Issues with explicit workarounds (NIM_MAX_MODEL_LEN, NIM_KVCACHE_PERCENT, LoRA rank caps) and statements that some issues have no workaround in 2.0.12.
- Limitations and uncertainty
- Vendor documentation of known issues; does not quantify incidence in customer fleets. Version-specific—verify against the deployed tag.
NIM Operator guidance requires inference metrics for HPA and long model-load startup probes
Official NIM Operator deployment guidance for 2.0.12 centers on NIMCache + NIMService, requires GPU Operator, PVC caching, and NGC secrets, and states CPU/memory metrics are not useful for HPA—use backend-native inference metrics (e.g., vllm:num_requests_waiting) via Prometheus Adapter. Startup probes allow up to ~20 minutes by default for model load; multi-node spec.args are not propagated on the LeaderWorkerSet path.
Why it matters, evidence and limitations
- Why it matters
- Campus and agency Kubernetes platforms cannot reuse generic HPA patterns for NIM; observability and probe budgets become acceptance criteria for production AI services.
- Evidence and measured results
- Opened docs detail prerequisites (K8s 1.26+, GPU Operator, storage, NGC key, Operator 3.0.2+), autoscaling example, health endpoint semantics (live vs ready), and troubleshooting for cache jobs and probe failures.
- Limitations and uncertainty
- Documentation guidance, not a measured multi-tenant SLED deployment. Metric names differ between vLLM and SGLang backends.
Independent write-up scopes NVIDIA’s 2.5x Nemotron 3 Ultra NIM capacity claim
An independent Sept 13, 2026 analysis reconstructs NVIDIA’s Nemotron 3 Ultra NIM 2.0.12 benchmark (4xB200; 64K/400 tokens; 76% KV-cache reuse; 50 TPS/user target; 718 to 1,997 tok/s) and concludes the result is unusually well-specified but remains a company-run, configuration-specific full-stack comparison that may not transfer to low-reuse or different latency targets.
Why it matters, evidence and limitations
- Why it matters
- Gives SLED evaluators a checklist to reproduce vendor capacity claims against local agentic workloads instead of accepting headline multipliers.
- Evidence and measured results
- Page dated Sep 13, 2026 tabulates hardware/workload/latency conditions, explains ~14 vs ~40 stream arithmetic, stresses interacting optimizations and 76% cache reuse, and states MustHave.ai did not independently reproduce the benchmark.
- Limitations and uncertainty
- Secondary analysis of NVIDIA’s Sept 10 blog (already archived elsewhere); not a new lab reproduction.
AI Enterprise EOL notices add immediate and January 2027 NIM migration deadlines
EOL notices list immediate End of Support for NIM Llama-3.1-70b-instruct (migrate to Llama-3.3-70b-instruct or Multi-LLM NIM), Morpheus, and Optimized TensorFlow 2, plus January 2027 deprecations for several PB6 model-specific NIMs (including Nemotron 3 Nano and Llama-3.3-Nemotron-Super-49B-v1.5) and removal of older GPUs from Infra 8.x with Infra 4.10 LTSB already EOL July 2026.
Why it matters, evidence and limitations
- Why it matters
- SLED estates still on Llama-3.1-70B NIM or older Infra LTSB branches face active support cliffs; education/gov platforms should schedule migrations before PB6 sunset.
- Evidence and measured results
- Opened EOL page tables Active Notices for deprecated and immediate End of Support components, with migration paths and action dates including Immediate and January 2027 (PB6).
- Limitations and uncertainty
- Vendor policy page; local entitlement and branch mapping still required. Some deprecation dates are future (Jan 2027).
NVIDIA production note: AI-Q/LangGraph agent scaling needed NIM CPU fix and phased rollout
NVIDIA engineers describe scaling an internal LangGraph deep-research agent (AI-Q blueprint) on OpenShift with local NIM: single-user profiling found Llama Nemotron Super 49B calls as the bottleneck; load tests at 10–50 concurrency informed hardware forecasts; a Helm misconfiguration starved a NIM of CPU at 100%; LLM timeouts required retries; phased rollout used NeMo Agent Toolkit OTEL + Datadog.
Why it matters, evidence and limitations
- Why it matters
- Provides a transferable operating pattern for SLED agent platforms—profile, load-test, fix config, phase rollout—while remaining an internal vendor deployment without public SLOs.
- Evidence and measured results
- Opened blog (dated Aug 27, 2025) details three-step process with NeMo Agent Toolkit profiler/sizing calculator, explicit CPU starvation finding, timeout handling snippet, and OTEL/Datadog monitoring during phased user expansion.
- Limitations and uncertainty
- Internal NVIDIA deployment; example sizing charts noted as toolkit sample not the actual agent data; dated Aug 2025 but newly relevant versus prior nvidia editions that did not cover this agent-scaling path.
How to read this edition
Source findings, measured results and limitations come from the cited publications. Patterns, operating questions, role takeaways and implementation considerations are Lighthouse Advisory interpretation, stated as questions to validate locally rather than guaranteed outcomes. Vendor and operator claims are labeled as claims. Full research method.
- Standards or public-body guidance
- Normative or advisory guidance from a standards body or public institution.
- Vendor claim
- A supplier-provided assertion that has not been upgraded to independent evidence.
- Independent reporting
- Independent reporting with attributable sources but without a formal evaluation design.
- Independent research
- Research conducted outside the implementing organization.