From the NVIDIA edition of November 10, 2026
Matching Workloads to Blackwell: Power-Resolved Benchmarking of NVIDIA B200 and B300 across HPC and AI Workloads
Mohsen Zand, Trung Dac Nguyen, H. Birali Runesha · HPC and AI performance evaluation research · United States (research computing center; two eight-GPU Blackwell clusters)
- Publisher
- arXiv
- Original publication
- Submitted October 30, 2026 (arXiv preprint; accepted at PMBS 2026 workshop at SC26)
- Source retrieved
- 2026-11-11
What happened
On two independent eight-GPU clusters, the authors find the B300 (Blackwell Ultra) is not a uniform improvement over the B200. It trails by up to 5.6x on convolutional inference, 8x on 3D vision transformer training, 24 to 27% on 70B LLM throughput and 16x on double-precision Quantum ESPRESSO, while winning on ViT-Large. On affected workloads the B300 draws only 36 to 53% of its 1100 W envelope, which the authors use to localize the cause to per-kernel cost rather than data supply. Two B300 nodes of identical silicon differ by 2.1x in time-to-first-token solely from a development vLLM build. They recommend per-workload procurement decisions benchmarked on the exact software build.
Why it matters
Directly useful to university research computing and public-sector HPC buyers choosing between Blackwell parts: it supplies independent, workload-level evidence and a method to test before buying. It reports no SLED deployment outcome.
Evidence and measured results
Abstract page read, plus the full HTML text for method and limitations. Harness, raw CSVs and plotting scripts are released. The paper states replication is uneven: CV and LLM results are single runs with spot-checked 2 to 4% variation; 3D ViT and HPC results are medians of five runs.
Limitations and uncertainty
Preprint accepted at a workshop; single-run CV and LLM measurements; harness keeps inputs in device memory so input-pipeline effects are excluded; cross-generation comparisons mix compiler and MPI versions; hardware access was provided by NVIDIA, Schmidt Sciences and Lenovo per the acknowledgments; the authors disclose LLM assistance in prose. Published October 30, newly relevant on Nov 10 because NVIDIA's same-day Snapshot post reports B200 and B300 results, and the paper shows how build and platform choices change measured outcomes.
Put this evidence to work
Lighthouse Advisory interpretation, grounded in this source. Enriched 2026-11-10; this does not change the original publication date. Each role follows the same checklist so takeaways compare across sources.
Sales
Role takeaway
- Customer problem
- Buyers cannot tell from specifications whether a newer GPU will help their specific workloads.
- Stakeholders
- Research computing director
- Principal investigators
- Procurement
- Facilities and power managers
- Discovery questions
- Which workloads dominate your GPU use?
- Can you access B200 and B300 systems for testing?
- Which software builds will run in production?
- Value hypothesis
- Per-workload benchmarking before purchase may avoid buying a part that underperforms on the center's codes.
- Potential engagement
- Pre-purchase benchmarking and procurement guidance engagement using the released harness.
- Unsupported claims
- That B300 is generally worse than B200
- That results apply to SLED workloads not tested
- That software gaps will close on a given date
Pre-sales engineering
Role takeaway
- Fit
- Fits research computing centers comparing Blackwell and prior-generation GPUs for mixed AI and HPC workloads.
- Architecture and integration
- Harness sweeps models, precisions, GPU counts and batch sizes with NVML board-power sampling at 20 Hz; results released as CSV.
- Prerequisites
- Access to B200 and B300 test systems
- Representative workloads
- Pinned CUDA, driver and library versions
- Constraints
- Single-run CV and LLM measurements with 2 to 4% variation
- Input pipeline excluded
- Compiler and MPI differences across generations
- Security
- Not assessed; use non-sensitive benchmark data.
- Proof of value
- Run released harness on candidate hardware
- Add power sampling to existing benchmarks
- Compare two software builds on identical hardware
Delivery
Role takeaway
- Work
- Short pre-purchase benchmarking engagement with fixed software stacks and repeated runs.
- Dependencies
- Hardware access
- Application owners
- Consistent builds
- Ownership
- Research computing group.
- Skills and adoption
- Performance engineering and profiling skills needed.
- Governance checkpoints
- Agree workloads and build versions before testing
- Review variance and replication
- Re-test after major software releases
- Acceptance criteria
- Proposed: per-workload results with repeated runs
- Proposed: documented software versions and power data
- Risks
- Test workloads unrepresentative
- Software changes alter results
- Access to hardware
Implementation considerations
Lighthouse Advisory interpretation across the operating dimensions a public-sector buyer must settle before this evidence becomes a design. Each note answers the question under its heading for this specific source.
Architecture and integration
What must connect, and where does the AI sit in the workflow?
Software stack maturity (CUDA 13.2, drivers, library and vLLM builds) can account for a large share of apparent hardware gaps; kernel-granularity and power profiling expose issues that specification sheets and throughput-only tests hide.
Governance
Who approves, reviews and stays accountable for outcomes?
Supports requiring workload-specific, reproducible benchmark evidence and named software builds in acquisition decisions, and re-testing as vendor software matures.
Security and privacy
What data, permissions and controls need testing?
Not applicable beyond noting that benchmarking used research workloads and not sensitive data.
Accessibility and workforce
Who is affected, and what skills or accommodations follow?
Kernel-level profiling and power analysis require specialized performance engineering skills; released harnesses and data lower the barrier for smaller centers.
Procurement
What should contracts, pricing and exit terms secure?
Supports writing acceptance tests tied to the buyer's own workloads and software builds, and avoiding purchase decisions based on specifications or vendor peak figures alone.
Operating model
Which teams own the service once it runs?
Suggests research computing centers keep a benchmarking function and a test cluster so procurement and software upgrades are validated on local workloads.
Publication history
- 2026-11-10NVIDIA · Issue 665 resources
Stable resource ID: arxiv-b200-b300-power-resolved-benchmarking-2026