{"resourceId":"arxiv-b200-b300-power-resolved-benchmarking-2026","versions":[{"version":"external-34a70bbc17fc01b4903f1ff3efc4036a05eca7dcaac81313ded13402745afe28","resource":{"id":"arxiv-b200-b300-power-resolved-benchmarking-2026","title":"Matching Workloads to Blackwell: Power-Resolved Benchmarking of NVIDIA B200 and B300 across HPC and AI Workloads","organization":"Mohsen Zand, Trung Dac Nguyen, H. Birali Runesha","sector":"HPC and AI performance evaluation research","geography":"United States (research computing center; two eight-GPU Blackwell clusters)","publishedAt":"Submitted October 30, 2026 (arXiv preprint; accepted at PMBS 2026 workshop at SC26)","publicationDate":"2026-10-30","eventDate":null,"sourceName":"arXiv","sourceLabel":"arXiv (academic research)","sourceUrl":"https://arxiv.org/abs/2610.41919","evidenceClass":"academic-research","outcomeClass":"mixed","topics":["infrastructure","governance-procurement"],"finding":"On two independent eight-GPU clusters, the authors find the B300 (Blackwell Ultra) is not a uniform improvement over the B200. It trails by up to 5.6x on convolutional inference, 8x on 3D vision transformer training, 24 to 27% on 70B LLM throughput and 16x on double-precision Quantum ESPRESSO, while winning on ViT-Large. On affected workloads the B300 draws only 36 to 53% of its 1100 W envelope, which the authors use to localize the cause to per-kernel cost rather than data supply. Two B300 nodes of identical silicon differ by 2.1x in time-to-first-token solely from a development vLLM build. They recommend per-workload procurement decisions benchmarked on the exact software build.","sledRelevance":"Directly useful to university research computing and public-sector HPC buyers choosing between Blackwell parts: it supplies independent, workload-level evidence and a method to test before buying. It reports no SLED deployment outcome.","evidence":"Abstract page read, plus the full HTML text for method and limitations. Harness, raw CSVs and plotting scripts are released. The paper states replication is uneven: CV and LLM results are single runs with spot-checked 2 to 4% variation; 3D ViT and HPC results are medians of five runs.","architectureImplications":"Software stack maturity (CUDA 13.2, drivers, library and vLLM builds) can account for a large share of apparent hardware gaps; kernel-granularity and power profiling expose issues that specification sheets and throughput-only tests hide.","governanceImplications":"Supports requiring workload-specific, reproducible benchmark evidence and named software builds in acquisition decisions, and re-testing as vendor software matures.","securityPrivacyImplications":"Not applicable beyond noting that benchmarking used research workloads and not sensitive data.","caveats":"Preprint accepted at a workshop; single-run CV and LLM measurements; harness keeps inputs in device memory so input-pipeline effects are excluded; cross-generation comparisons mix compiler and MPI versions; hardware access was provided by NVIDIA, Schmidt Sciences and Lenovo per the acknowledgments; the authors disclose LLM assistance in prose. Published October 30, newly relevant on Nov 10 because NVIDIA's same-day Snapshot post reports B200 and B300 results, and the paper shows how build and platform choices change measured outcomes.","streamIds":["nvidia"],"roles":{"sales":"Use this paper to reset expectations: newer does not mean uniformly faster, so quote per-workload evidence. For research computing prospects, propose testing their own codes on B200 and B300 before committing to a part, and note the paper's finding that software builds changed time-to-first-token by 2.1x. Acknowledge the study is independent but limited in replication. Avoid claiming either part is better in general. This is interpretation of the paper's procurement guidance.","engineering":"The practical takeaway is to profile at kernel granularity with board power, not throughput alone, and to pin the exact software build you will deploy. The authors show B300 draws 36 to 53% of its power envelope on affected convolutional workloads, pointing at per-kernel cost. Reuse their released harness to check your workloads, and repeat single-run results given 2 to 4% variation. Expect software maturity to shift results over time, so re-measure after driver and library updates.","delivery":"Delivery would run a short benchmarking engagement before purchase: select representative codes, fix the software stack, measure throughput, power and variance on candidate parts, and report per-workload results. Dependencies are access to candidate hardware, application owners and consistent builds. Ownership sits with the research computing group. Acceptance criteria should require repeated runs and documented software versions; plan to re-test after major releases."},"roleDetails":{"sales":{"customerProblem":"Buyers cannot tell from specifications whether a newer GPU will help their specific workloads.","stakeholders":["Research computing director","Principal investigators","Procurement","Facilities and power managers"],"discoveryQuestions":["Which workloads dominate your GPU use?","Can you access B200 and B300 systems for testing?","Which software builds will run in production?"],"valueHypothesis":"Per-workload benchmarking before purchase may avoid buying a part that underperforms on the center's codes.","potentialEngagement":"Pre-purchase benchmarking and procurement guidance engagement using the released harness.","unsupportedClaims":["That B300 is generally worse than B200","That results apply to SLED workloads not tested","That software gaps will close on a given date"]},"engineering":{"fit":"Fits research computing centers comparing Blackwell and prior-generation GPUs for mixed AI and HPC workloads.","architectureIntegration":"Harness sweeps models, precisions, GPU counts and batch sizes with NVML board-power sampling at 20 Hz; results released as CSV.","prerequisites":["Access to B200 and B300 test systems","Representative workloads","Pinned CUDA, driver and library versions"],"constraints":["Single-run CV and LLM measurements with 2 to 4% variation","Input pipeline excluded","Compiler and MPI differences across generations"],"security":"Not assessed; use non-sensitive benchmark data.","proofOfValue":["Run released harness on candidate hardware","Add power sampling to existing benchmarks","Compare two software builds on identical hardware"]},"delivery":{"work":"Short pre-purchase benchmarking engagement with fixed software stacks and repeated runs.","dependencies":["Hardware access","Application owners","Consistent builds"],"ownership":"Research computing group.","skillsAdoption":"Performance engineering and profiling skills needed.","governanceCheckpoints":["Agree workloads and build versions before testing","Review variance and replication","Re-test after major software releases"],"acceptanceCriteria":["Proposed: per-workload results with repeated runs","Proposed: documented software versions and power data"],"risks":["Test workloads unrepresentative","Software changes alter results","Access to hardware"]}},"retrievedAt":"2026-11-10T21:59:07-06:00","enrichedAt":"2026-11-10T22:02:26-06:00","enrichmentBasis":"retrieved source","accessibilityWorkforceImplications":"Kernel-level profiling and power analysis require specialized performance engineering skills; released harnesses and data lower the barrier for smaller centers.","procurementImplications":"Supports writing acceptance tests tied to the buyer's own workloads and software builds, and avoiding purchase decisions based on specifications or vendor peak figures alone.","operatingModelImplications":"Suggests research computing centers keep a benchmarking function and a test cluster so procurement and software upgrades are validated on local workloads.","peerReview":"preprint","sourceVerification":{"openedUrl":"https://arxiv.org/abs/2610.41919","referenceExcerpt":"The B300 is not a uniform successor to the B200","promptVersion":"sled-research-v4.0","model":null,"basis":"agent-reported inspection"}}}]}