{"resourceId":"nvidia-cc-hardware-rooted-security-benchmark-20260702","versions":[{"version":"external-05001af4557e8c4147b1b14c234207d47281aad3264d5285ac85b00b80923caf","resource":{"id":"nvidia-cc-hardware-rooted-security-benchmark-20260702","title":"NVIDIA claims Confidential Computing inference overhead roughly 1–8% on HGX B300","organization":"NVIDIA","sector":"Confidential computing / AI security","geography":"Global (vendor benchmark)","publishedAt":"July 2, 2026","publicationDate":"2026-07-02","eventDate":null,"sourceName":"NVIDIA Technical Blog","sourceLabel":"Vendor benchmark / best practice","sourceUrl":"https://developer.nvidia.com/blog/hardware-rooted-ai-security-that-wont-slow-you-down/","evidenceClass":"vendor-claim","outcomeClass":"emerging","topics":["data-security","infrastructure","developers-agents"],"finding":"NVIDIA reports Blackwell Confidential Computing (RTX PRO 6000, HGX B200/B300) with NRAS attestation and NVLink encryption on multi-GPU HGX, stating inference performance up to 98% of non-CC. On HGX B300 with Qwen 3.5 397B-A17B FP8 in SGLang, tabulated delta vs CC-off ranges about -1% to -8% for throughput/TPOT across concurrencies 4–256. Optimizations cited: FlashInfer CC-safe autotuner, SGLang async D2H worker, piecewise CUDA graphs.","sledRelevance":"Attractive for regulated SLED inference needing in-use protection. Claims are vendor-measured on a CC-aware SGLang stack—must be cross-checked against independent results showing much higher overhead for CC-naive engines.","evidence":"Blog dated July 2, 2026 provides Table 1 relative performance for Qwen3.5-397B-A17B-FP8 on HGX B300/Intel TDX; software stack lists driver 595.71.05, CUDA 13.2, SGLang v0.5.12-cu130; states attestation is typically one-time at startup.","architectureImplications":"Plan CVM+GPU-CC with CC-aware engines (SGLang/TRT-LLM patches), NRAS attestation workflow, and CPU pinning guidance referenced in the post.","governanceImplications":"Maps rhetorically to NIST ZTA / HIPAA / GDPR Art. 32 links in resources list; does not constitute a compliance certification.","securityPrivacyImplications":"Hardware root of trust and remote attestation before secret deployment; protects data/model in use on GPU.","caveats":"Single vendor model/engine/config; steady-state inference focus; does not report training or CC-naive engine penalties highlighted elsewhere.","streamIds":["nvidia"],"roles":{"sales":"Sell CC as in-use protection with bounded overhead only when the serving stack is CC-aware. Do not quote 98% without naming model, engine, and concurrency.","engineering":"Require SGLang/TRT-LLM CC patches, NRAS integration, and TDX/SEV-SNP host config; benchmark the prospect's model card under CC-on vs off.","delivery":"Sequence attestation onboarding, engine upgrade, and dual-run performance acceptance before cutting over regulated workloads."},"roleDetails":{"sales":{"customerProblem":"Regulated workloads need confidentiality during inference without abandoning GPU performance.","stakeholders":["CISO","AI platform","compliance","cloud/hosting"],"discoveryQuestions":["Which engine (SGLang/TRT-LLM/vLLM) is in production?","Is attestation (NRAS) required before model keys?","What overhead budget is acceptable?"],"valueHypothesis":"On CC-aware stacks, overhead can stay in low single digits for steady-state decode on Blackwell Ultra.","potentialEngagement":"CC-on/off benchmark on the customer's top model with attestation dry-run.","unsupportedClaims":["Universal 98% for all engines","Compliance certification","Zero host/device bandwidth impact"]},"engineering":{"fit":"Fit when Intel TDX/AMD SEV-SNP + Blackwell CC + CC-aware engine are available.","architectureIntegration":"CVM with GPU passthrough, NRAS attestation, SGLang/FlashInfer CC optimizations.","prerequisites":["CC-capable GPUs/driver","CPU TEE","CC-aware inference engine"],"constraints":["Secure launch overhead on small work","Reduced H2D bandwidth sensitivity","Vendor-reported config"],"security":"Fused GPU signing key + NRAS RIM verification before secrets.","proofOfValue":["Delta throughput/TPOT within agreed band","Successful attestation handshake","No secret deploy without verify"]},"delivery":{"work":"TEE host bring-up, engine patch rollout, attestation ops, dual-run tests.","dependencies":["hypervisor/TEE team","model owners","key management"],"ownership":"Security architecture + inference platform jointly.","skillsAdoption":"Operators need attestation runbooks.","governanceCheckpoints":["Threat-model review","Key ceremony under attestation","Performance gate"],"acceptanceCriteria":["Documented single-digit-class overhead on agreed card","Attestation evidence retained","Rollback to non-CC path tested"],"risks":["Stack not CC-aware yields much higher real overhead","Misread marketing as compliance"]}},"retrievedAt":"2026-09-17T03:14:00Z","enrichedAt":"2026-09-17T03:20:00Z","enrichmentBasis":"retrieved source","accessibilityWorkforceImplications":"No direct accessibility impacts; security posture may unlock use of sensitive datasets by authorized staff.","procurementImplications":"Specify CC-aware engine versions and attestation service in SOWs; avoid overhead SLAs without workload cards.","operatingModelImplications":"Adds attestation and TEE fleet management to inference ops.","sourceVerification":{"openedUrl":"https://developer.nvidia.com/blog/hardware-rooted-ai-security-that-wont-slow-you-down/","referenceExcerpt":"enabling Confidential Computing produces minimal throughput and time-per-output-token overhead","promptVersion":"sled-research-v4.0","model":null,"basis":"agent-reported inspection"}}}]}