{"resourceId":"tinfoil-nvidia-cc-overhead-blackwell-20260623","versions":[{"version":"external-05001af4557e8c4147b1b14c234207d47281aad3264d5285ac85b00b80923caf","resource":{"id":"tinfoil-nvidia-cc-overhead-blackwell-20260623","title":"Independent Tinfoil benchmarks: Blackwell CC overhead from ~1% to >100% by engine","organization":"Tinfoil","sector":"Confidential computing / independent evaluation","geography":"Global (independent lab blog)","publishedAt":"June 23, 2026","publicationDate":"2026-06-23","eventDate":null,"sourceName":"Tinfoil Blog","sourceLabel":"Independent technical evaluation","sourceUrl":"https://tinfoil.sh/blog/2026-06-23-confidential-computing-overhead","evidenceClass":"independent-research","outcomeClass":"cautionary","topics":["data-security","infrastructure","developers-agents"],"finding":"Tinfoil's public Blackwell/Hopper CC study (Intel TDX + NVIDIA CC) finds GPU math near-free but bounce-buffer and sync taxes dominate: ITL overhead ~1% with CC-aware TRT-LLM/SGLang to ~30%+ with CC-naive vLLM in compute-bound regimes and >100% at low concurrency; training throughput loss 18–32% on NeMo AutoModel; pinned H2D collapses to ~0.20x under CC while pageable keeps ~0.78x; kernel launch 3.5x slower on B300; Blackwell multi-GPU loses multicast/NVLS.","sledRelevance":"Primary independent counterweight to vendor won't-slow-you-down messaging. For SLED regulated inference, engine choice and CUDA-graph discipline become security-performance controls—not optional tuning.","evidence":"Long-form post Jun 23, 2026 details 8xB300 and 8xH200 TDX setups, engine versions, and tables for LoRA vs full SFT training losses (18–32%), ITL overhead by concurrency, and microbenchmarks (kernel launch, pinned vs pageable bandwidth).","architectureImplications":"Prefer CC-aware engines, CUDA graphs (never eager), narrower TP with more replicas for latency paths, and pageable paths for H2D under CC; re-profile speculative decoding.","governanceImplications":"Document that lift-and-shift CC is not performance-neutral; require CC-aware acceptance tests in security projects.","securityPrivacyImplications":"Confirms NVLink encryption on Blackwell improves security posture while exposing fixed per-exchange tolls and multicast disablement.","caveats":"Vendor-of-CC-services (Tinfoil) with incentive to publish; still the clearest public multi-engine Blackwell CC map. Some root causes marked as under discussion with NVIDIA.","streamIds":["nvidia"],"roles":{"sales":"Use Tinfoil to bound CC deals: overhead is engine- and concurrency-dependent. Push back on blanket low-overhead claims when the estate runs stock vLLM.","engineering":"Build CC performance budgets from bounce-buffer mechanics; mandate CC-aware patches and graph mode in the reference architecture.","delivery":"Add CC-on micro and end-to-end benches to delivery gates; train teams not to enable pinned-memory optimizations under CC."},"roleDetails":{"sales":{"customerProblem":"Security teams mandate CC while platform teams fear unexplained slowdowns.","stakeholders":["CISO","platform eng","FinOps"],"discoveryQuestions":["Is the engine CC-aware or stock vLLM?","What concurrency is typical?","Is training in scope?"],"valueHypothesis":"With CC-aware stacks and high concurrency, overhead can approach vendor low-single-digit claims; naive stacks will not.","potentialEngagement":"Independent-style CC overhead assessment across the customer's engines.","unsupportedClaims":["CC is always ~1% slowdown","Training overhead equals inference","Pinned memory remains best practice under CC"]},"engineering":{"fit":"Essential scrutiny input for any Blackwell CC design.","architectureIntegration":"Select TRT-LLM/SGLang CC paths; avoid eager; reconsider TP width; watch multicast-dependent kernels.","prerequisites":["CC-aware engine builds","CUDA graphs","Workload concurrency plan"],"constraints":["Bounce buffer taxes","3–4x kernel launch cost","Multicast disabled on Blackwell CC"],"security":"Align threat model with encrypted CPU-GPU and NVLink paths.","proofOfValue":["CC-on vs off ITL within budget","Pinned vs pageable H2D check","No eager mode in prod"]},"delivery":{"work":"Engine hardening, bench harness, and operator education on CC pitfalls.","dependencies":["security architecture","inference eng","NVIDIA support"],"ownership":"Platform engineering with security co-sign.","skillsAdoption":"Teams must learn CC-specific performance debugging.","governanceCheckpoints":["Security exception if naive engine retained","Bench sign-off","Training-vs-inference scope split"],"acceptanceCriteria":["Published overhead table for estate engines","Runbook banning eager under CC","Training loss expectations documented"],"risks":["Stakeholders only read vendor 98% claim","Low-concurrency interactive apps look 2x worse"]}},"retrievedAt":"2026-09-17T03:14:00Z","enrichedAt":"2026-09-17T03:20:00Z","enrichmentBasis":"retrieved source","accessibilityWorkforceImplications":"Performance cliffs at low concurrency can degrade interactive staff tools if CC is enabled without tuning.","procurementImplications":"Write CC overhead ranges by engine into contracts; require bidder benches under CC-on.","operatingModelImplications":"Security enablement becomes a continuous performance engineering practice.","sourceVerification":{"openedUrl":"https://tinfoil.sh/blog/2026-06-23-confidential-computing-overhead","referenceExcerpt":"GPU math is free. Crossing an encrypted link, whether with data, a command, or just a synchronization signal, is taxed in CC mode.","promptVersion":"sled-research-v4.0","model":null,"basis":"agent-reported inspection"}}}]}