From the NVIDIA edition of September 16, 2026
NVIDIA claims Confidential Computing inference overhead roughly 1–8% on HGX B300
NVIDIA · Confidential computing / AI security · Global (vendor benchmark)
- Publisher
- NVIDIA Technical Blog
- Original publication
- July 2, 2026
- Source retrieved
- 2026-09-17
What happened
NVIDIA reports Blackwell Confidential Computing (RTX PRO 6000, HGX B200/B300) with NRAS attestation and NVLink encryption on multi-GPU HGX, stating inference performance up to 98% of non-CC. On HGX B300 with Qwen 3.5 397B-A17B FP8 in SGLang, tabulated delta vs CC-off ranges about -1% to -8% for throughput/TPOT across concurrencies 4–256. Optimizations cited: FlashInfer CC-safe autotuner, SGLang async D2H worker, piecewise CUDA graphs.
Why it matters
Attractive for regulated SLED inference needing in-use protection. Claims are vendor-measured on a CC-aware SGLang stack—must be cross-checked against independent results showing much higher overhead for CC-naive engines.
Evidence and measured results
Blog dated July 2, 2026 provides Table 1 relative performance for Qwen3.5-397B-A17B-FP8 on HGX B300/Intel TDX; software stack lists driver 595.71.05, CUDA 13.2, SGLang v0.5.12-cu130; states attestation is typically one-time at startup.
Limitations and uncertainty
Single vendor model/engine/config; steady-state inference focus; does not report training or CC-naive engine penalties highlighted elsewhere.
Put this evidence to work
Lighthouse Advisory interpretation, grounded in this source. Enriched 2026-09-17; this does not change the original publication date. Each role follows the same checklist so takeaways compare across sources.
Sales
Role takeaway
- Customer problem
- Regulated workloads need confidentiality during inference without abandoning GPU performance.
- Stakeholders
- CISO
- AI platform
- compliance
- cloud/hosting
- Discovery questions
- Which engine (SGLang/TRT-LLM/vLLM) is in production?
- Is attestation (NRAS) required before model keys?
- What overhead budget is acceptable?
- Value hypothesis
- On CC-aware stacks, overhead can stay in low single digits for steady-state decode on Blackwell Ultra.
- Potential engagement
- CC-on/off benchmark on the customer's top model with attestation dry-run.
- Unsupported claims
- Universal 98% for all engines
- Compliance certification
- Zero host/device bandwidth impact
Pre-sales engineering
Role takeaway
- Fit
- Fit when Intel TDX/AMD SEV-SNP + Blackwell CC + CC-aware engine are available.
- Architecture and integration
- CVM with GPU passthrough, NRAS attestation, SGLang/FlashInfer CC optimizations.
- Prerequisites
- CC-capable GPUs/driver
- CPU TEE
- CC-aware inference engine
- Constraints
- Secure launch overhead on small work
- Reduced H2D bandwidth sensitivity
- Vendor-reported config
- Security
- Fused GPU signing key + NRAS RIM verification before secrets.
- Proof of value
- Delta throughput/TPOT within agreed band
- Successful attestation handshake
- No secret deploy without verify
Delivery
Role takeaway
- Work
- TEE host bring-up, engine patch rollout, attestation ops, dual-run tests.
- Dependencies
- hypervisor/TEE team
- model owners
- key management
- Ownership
- Security architecture + inference platform jointly.
- Skills and adoption
- Operators need attestation runbooks.
- Governance checkpoints
- Threat-model review
- Key ceremony under attestation
- Performance gate
- Acceptance criteria
- Documented single-digit-class overhead on agreed card
- Attestation evidence retained
- Rollback to non-CC path tested
- Risks
- Stack not CC-aware yields much higher real overhead
- Misread marketing as compliance
Implementation considerations
Lighthouse Advisory interpretation across the operating dimensions a public-sector buyer must settle before this evidence becomes a design. Each note answers the question under its heading for this specific source.
Architecture and integration
What must connect, and where does the AI sit in the workflow?
Plan CVM+GPU-CC with CC-aware engines (SGLang/TRT-LLM patches), NRAS attestation workflow, and CPU pinning guidance referenced in the post.
Governance
Who approves, reviews and stays accountable for outcomes?
Maps rhetorically to NIST ZTA / HIPAA / GDPR Art. 32 links in resources list; does not constitute a compliance certification.
Security and privacy
What data, permissions and controls need testing?
Hardware root of trust and remote attestation before secret deployment; protects data/model in use on GPU.
Accessibility and workforce
Who is affected, and what skills or accommodations follow?
No direct accessibility impacts; security posture may unlock use of sensitive datasets by authorized staff.
Procurement
What should contracts, pricing and exit terms secure?
Specify CC-aware engine versions and attestation service in SOWs; avoid overhead SLAs without workload cards.
Operating model
Which teams own the service once it runs?
Adds attestation and TEE fleet management to inference ops.
Publication history
- 2026-09-16NVIDIA · Issue 119 resources
Stable resource ID: nvidia-cc-hardware-rooted-security-benchmark-20260702