{"resourceId":"arxiv-cc-blackwell-benchmark-260826575","versions":[{"version":"external-065051557cb29b36d65bba0d4456f1ee4fc2aecd8caa7f1eb3df9235436033be","resource":{"id":"arxiv-cc-blackwell-benchmark-260826575","title":"arXiv: Benchmarking Confidential Computing on Blackwell—~1–3% when configured, 30–40% when not","organization":"Confidential.ai (Asad & Grunseid)","sector":"Academic / independent systems research","geography":"Lab study (single-host B200)","publishedAt":"arXiv 2608.26575v2, 01 Sep 2026 (dated June 2026)","publicationDate":"2026-09-01","eventDate":"2026-06-01","sourceName":"arXiv","sourceLabel":"Academic performance study","sourceUrl":"https://arxiv.org/html/2608.26575","evidenceClass":"academic-research","outcomeClass":"mixed","topics":["data-security","infrastructure","developers-agents"],"finding":"Paired CC-on vs CC-off measurements on one Xeon + 8x B200 host under Intel TDX + NVIDIA Confidential Computing find well-configured confidential inference at ~1–3% throughput tax (e.g., MoE TP4 ~1.5%, TP8 ~3%, single-GPU with full CUDA graphs + patches <1%), while stock/eager stacks see ~30–40% penalties from avoidable host-submission and bounce-buffer costs. GPU compute, energy draw, and usable memory largely unaffected (fixed ~700 MB carveout). Training on 8 GPUs costs ~10–13% via encrypted collectives. Authors localize cost to encrypted PCIe/command-path and NVLE; cross-node RDMA/disagg not measured.","sledRelevance":"Highly relevant for SLED regulated inference: updates the prior Tinfoil/NVIDIA CC debate with controlled Blackwell methods. Shows CC can be near-parity if frameworks are CC-aware—procurement must specify engine/build/concurrency, not a single overhead percentage.","evidence":"Controlled same-host comparisons; MiniMax-M2.7 MoE medians; vLLM piecewise-graph ~185 submissions predicting ~2.28 ms/step overhead; eager vs full-graph probe 2.41x to 1.04x; NVLE collective tax; training 10–13%; limitations: no security audit, no cross-node measurement, attestation startup not timed.","architectureImplications":"Prefer full CUDA graphs / CC-patched frameworks, decode-heavy profiles, and EP/DP over excess TP to cut NVLE; budget training CC separately; reboot hygiene for clean benches.","governanceImplications":"RFPs should require CC-on vs CC-off paired results on the buyer model/framework—not brochure averages.","securityPrivacyImplications":"Paper measures performance only; does not validate attestation or adversary resistance.","caveats":"Single-host lab; framework versions dated; cross-node disagg out of scope; authors affiliated with Confidential.ai.","streamIds":["nvidia"],"roles":{"sales":"For regulated buyers demanding confidential GPUs, lead with overhead is configuration-scoped: cite ~1–3% well-tuned versus ~30–40% naive stacks from this Blackwell study. Offer a paired CC-on/off PoC on the customer serving engine rather than quoting a single tax. Explicitly exclude unmeasured cross-node disagg from any sales guarantee. Keep claims bounded to the inspected evidence and require site-matched proof before any capacity or interconnect commitment.","engineering":"Fit when TDX+CC is mandatory. Integrate CC-aware SGLang/vLLM builds, full graphs, and TP matched to need. Prerequisites: CC-capable host/GPU firmware, attestation path, clean reboot methodology. Proof of value: paired throughput/TTFT/TPOT on the production model card under CC-on and CC-off. Document configuration pins and failure modes so the next upgrade cannot silently invalidate the measured result. Document configuration pins and failure modes so the next upgrade cannot silently invalidate the measured result.","delivery":"Work includes CC enablement, framework patches, and regression benches after every stack change. Ownership: platform security plus inference SRE. Risks: eager mode regressions, NVLE tax under high TP, and assuming single-host results transfer to multi-node Dynamo without new measurement. Record acceptance evidence in the program archive so later editions can cite what was actually proven. Record acceptance evidence in the program archive so later editions can cite what was actually proven."},"roleDetails":{"sales":{"customerProblem":"Regulated workloads need CC but fear unacceptable inference tax.","stakeholders":["CISO","AI platform","procurement"],"discoveryQuestions":["Which engine build is CC-aware?","Is serving single-host or multi-node disagg?","Training under CC required?"],"valueHypothesis":"With CC-aware stacks, Blackwell inference tax can stay low single digits on matched workloads.","potentialEngagement":"Two-week paired CC benchmark on customer model/framework.","unsupportedClaims":["1% for every stack","Multi-node disagg tax known","Security attested by this paper"]},"engineering":{"fit":"Strong for single-host/multi-GPU CC inference; cautious for unmeasured cross-node paths.","architectureIntegration":"TDX guest + GPU CC bit; prefer graphs; minimize unnecessary TP; separate training budget.","prerequisites":["CC firmware","patched framework","attestation workflow","clean-boot bench harness"],"constraints":["Single-host evidence","framework version sensitivity","NVLE floor under collectives"],"security":"Performance study only—pair with attestation/policy controls elsewhere.","proofOfValue":["Paired tok/s within target tax","TTFT/TPOT hold","Training tax itemized if needed"]},"delivery":{"work":"Enable CC, patch serving stack, run paired benches, document config pins.","dependencies":["OEM firmware","framework maintainers","security attestation"],"ownership":"Platform security engineering with inference SRE.","skillsAdoption":"On-call must recognize eager/CC misconfig as 30%+ regressions.","governanceCheckpoints":["No production CC without paired bench","Disagg claims deferred","Config pins in change control"],"acceptanceCriteria":["Tax within agreed bound","Runbook for CC misconfig","Attestation green"],"risks":["Naive stack penalties","Multi-node surprise","Profiler/timing caveats under CC"]}},"retrievedAt":"2026-09-20T03:06:30Z","enrichedAt":"2026-09-20T03:13:48Z","enrichmentBasis":"retrieved source","accessibilityWorkforceImplications":"None stated.","procurementImplications":"Specify CC-aware engine versions and paired-bench acceptance in contracts.","operatingModelImplications":"CC becomes a continuous configuration discipline, not a one-time checkbox.","sourceVerification":{"openedUrl":"https://arxiv.org/html/2608.26575","referenceExcerpt":"Confidential inference on Blackwell achieves low single-digit throughput overhead when the stack is configured correctly, at about 1-3%.","promptVersion":"sled-research-v4.0","model":null,"basis":"agent-reported inspection"}}}]}