From the NVIDIA edition of September 17, 2026
Independent read: Vera Rubin MLPerf gains are real but cost-per-token still unproven
DIY AI · Independent AI infrastructure analysis · Global (English-language analysis)
- Publisher
- DIY AI
- Original publication
- September 16, 2026
- Source retrieved
- 2026-09-18
What happened
Steven Jones (Sep 16, 2026) cross-checks public MLPerf entries and finds Vera Rubin NVL72 vs like-sized GB300 gains ranging roughly 1.74x–3.74x by workload/scenario (largest on Qwen3-VL Interactive ~3.74x; DeepSeek-R1 Interactive ~2.58x; Offline/Server often ~1.7–2.0x). Notes submissions are not MLPerf power results, lists nameplate TGP sums (72×2300W vs 72×1400W) as non-authoritative, stresses software stack differences (TensorRT-LLM vs vLLM+Dynamo; FP4 vs BF16), preview status, and that cost-per-useful-token remains unproven without wall power and pricing.
Why it matters
Directly usable scrutiny for SLED procurement: prevents treating the vendor 3.7x headline as a blanket refresh ROI and supplies a buyer request list (wall power, matched scenarios, sustained throughput, software pins).
Evidence and measured results
Article publishes a six-row comparison table from public MLPerf entries for VR200 NVL72 vs GB300 NVL72; states Rubin results are not power submissions; flags preview marking; contrasts 512-GPU AMD aggregate leaderboard vs per-GPU normalization; lists buyer follow-ups including measured wall power and migration economics vs existing Blackwell.
Limitations and uncertainty
Secondary analysis of public tables; nameplate TGP arithmetic is illustrative not measured rack power; author incentives are independent-media not lab audit.
Put this evidence to work
Lighthouse Advisory interpretation, grounded in this source. Enriched 2026-09-18; this does not change the original publication date. Each role follows the same checklist so takeaways compare across sources.
Sales
Role takeaway
- Customer problem
- Headline 3.7x figures risk overselling refresh ROI to budget owners.
- Stakeholders
- CFO/budget
- procurement
- platform engineering
- facilities
- Discovery questions
- Which MLPerf scenario matches your SLO?
- Do you have wall-power metering on the candidate rack?
- What happens to existing Blackwell contracts?
- Value hypothesis
- Independent scenario decomposition prevents bad refresh commitments while preserving a credible throughput narrative.
- Potential engagement
- Cost-per-useful-token evaluation scoped to one production model card.
- Unsupported claims
- MLPerf proves cheaper tokens
- All scenarios see 3.7x
- Immediate fleet replacement justified
Pre-sales engineering
Role takeaway
- Fit
- Fit for teams with instrumentation and access to result tables; weak fit if only marketing briefings are available.
- Architecture and integration
- Instrument wall power, pin serving stack, and compare like-sized racks under buyer latency targets.
- Prerequisites
- Power metering
- Access to MLPerf entries
- Production traffic profile
- Constraints
- No official power submission
- Preview availability
- Stack-specific scores
- Security
- N/A in source—do not infer.
- Proof of value
- Scenario table reproduced
- Wall-power capture during run
- Software pin list attached
Delivery
Role takeaway
- Work
- Convert scrutiny checklist into test plan and decision gate.
- Dependencies
- facilities metering
- vendor SE
- finance
- Ownership
- AI infrastructure program manager.
- Skills and adoption
- Analysts need MLPerf scenario and TCO literacy.
- Governance checkpoints
- No-savings-without-power rule
- Preview hardware policy
- Blackwell residual review
- Acceptance criteria
- Matched scenario results filed
- Power+price inputs present or claim withdrawn
- Migration analysis documented
- Risks
- Vendors decline power data
- Preview slips
- Finance still anchors on 3.7x
Implementation considerations
Lighthouse Advisory interpretation across the operating dimensions a public-sector buyer must settle before this evidence becomes a design. Each note answers the question under its heading for this specific source.
Architecture and integration
What must connect, and where does the AI sit in the workflow?
Size to the scenario closest to production concurrency; do not copy a single batch size from the headline; require disaggregation/quantization details to reproduce.
Governance
Who approves, reviews and stays accountable for outcomes?
Document that MLPerf throughput ≠ authorized cost case; keep Blackwell depreciation analysis explicit.
Security and privacy
What data, permissions and controls need testing?
No security findings; economic scrutiny only.
Accessibility and workforce
Who is affected, and what skills or accommodations follow?
Clearer procurement literacy reduces wasted staff time on mismatched refresh projects.
Procurement
What should contracts, pricing and exit terms secure?
Attach wall-power, matched-scenario, and software-pin requirements to Rubin evaluations.
Operating model
Which teams own the service once it runs?
Keeps Blackwell and Rubin as portfolio choices rather than automatic replacement.
Publication history
- 2026-09-17NVIDIA · Issue 129 resources
Stable resource ID: diyai-vera-rubin-mlperf-procurement-gap-20260916