{"resourceId":"serialized-bridge-blackwell-gpu-cc-260623969","versions":[{"version":"external-7c313db30ef4d7c733c30cd6f948ff8afb8c4b82bb4e4172a242b3fd9a0777fb","resource":{"id":"serialized-bridge-blackwell-gpu-cc-260623969","title":"Independent study: Blackwell GPU-CC LLM losses come from serialized VM-GPU bridge","organization":"Phala Network researchers (Hang Yin, Kevin Wang)","sector":"Independent / academic systems research","geography":"Global (Blackwell platforms measured: RTX Pro 6000, B300 HGX)","publishedAt":"arXiv 2606.23969v2, 09 Jul 2026 (May 2026 paper)","publicationDate":"2026-07-09","eventDate":null,"sourceName":"arXiv cs.DC preprint","sourceLabel":"Independent research preprint","sourceUrl":"https://arxiv.org/html/2606.23969v2","evidenceClass":"independent-research","outcomeClass":"mixed","topics":["infrastructure","data-security","developers-agents"],"finding":"On Blackwell, GPU-local compute under confidential computing is near parity (BF16 matmul ~0.998x), but LLM serving under Intel TDX + GPU-CC still loses 13–27% throughput; KV-cache restore can more than double. Authors attribute losses to a serialized, high-setup-cost CVM–GPU bridge and show runtime mitigations (e.g., --no-async-scheduling recovers ~57%; worker-thread drain up to ~92% in qualified runs). They also note a fabric-attestation gap for confidential multi-GPU NVSwitch tenants.","sledRelevance":"Critical for SLED regulated workloads that assume confidential GPUs erase the performance tax. Agencies should budget serving-path engineering and attestation gaps, not only encrypted VRAM marketing claims.","evidence":"Abstract and introduction report B300 matmul parity, 13–27% serving loss, +131% KV-restore, 34x model-load slowdown with default loader; recovery hierarchy and remaining Fabric Manager/NVSwitch attestation gap are documented in the opened HTML preprint.","architectureImplications":"Design confidential serving stacks around bridge serialization: fewer small H2D/D2H crossings, CC-aware loaders, and scheduler policies that do not assume free async DMA.","governanceImplications":"Procurement language for near-native confidential performance must specify workload class (matmul vs LLM serving) and require attestation scope including fabric components where multi-GPU tenancy is claimed.","securityPrivacyImplications":"Confidential multi-GPU tenants can keep NVLink P2P inside a CVM, but Fabric Manager identity and NVSwitch routing state remain outside tenant attestation evidence per the paper.","caveats":"Preprint (not peer-reviewed journal version here). Results are on specific Blackwell platforms and vLLM-centered stacks; mitigations are qualified and may not transfer unchanged.","streamIds":["nvidia","research"],"roles":{"sales":"When buyers require GPU-CC for regulated data, set expectations that serving overheads remain material unless runtimes are retuned; cite independent bridge findings versus GPU-local parity claims.","engineering":"Profile H2D/D2H under CC, disable harmful async scheduling defaults, pool secure contexts for model load, and validate KV offload paths.","delivery":"Include CC-specific performance acceptance tests (decode throughput, KV restore, model load) and attestation boundary review for multi-GPU fabrics."},"roleDetails":{"sales":{"customerProblem":"Regulated SLED workloads need confidential GPUs but still must meet latency SLOs.","stakeholders":["CISO","platform engineering","procurement","research computing"],"discoveryQuestions":["Which workloads will run under GPU-CC?","What throughput tax is acceptable?","Does attestation need to cover fabric?"],"valueHypothesis":"Honest scoping of bridge overheads plus runtime mitigations preserves trust while enabling confidential on-prem/cloud serving.","potentialEngagement":"Proof-of-value on representative LLM serving under CC with before/after scheduler patches.","unsupportedClaims":["Zero overhead confidential LLM serving","Attestation covers entire NVSwitch fabric by default"]},"engineering":{"fit":"Fit for confidential LLM serving on Blackwell when runtime is redesigned for serialized secure copies.","architectureIntegration":"Integrate TDX/GPU-CC with vLLM/NIM serving; apply scheduling and loader changes; plan multi-GPU partition vocabulary.","prerequisites":["CC-capable GPUs and CPU TEE","Measurement harness","Patchable serving runtime","Attestation pipeline"],"constraints":["Async DMA assumptions break","Small copies pay high setup cost","Fabric attestation incomplete"],"security":"Threat model must include bridge metadata and fabric control-plane trust.","proofOfValue":["Measured decode gap vs baseline","KV restore within target","Model load time after CC-aware loader"]},"delivery":{"work":"CC enablement, runtime hardening, and acceptance testing for confidential serving.","dependencies":["platform security","inference platform team","vendor support"],"ownership":"Security owns attestation policy; platform owns runtime performance.","skillsAdoption":"Engineers need CC debugging skills beyond standard CUDA profiling.","governanceCheckpoints":["Threat-model review","Perf gate under CC","Attestation evidence review"],"acceptanceCriteria":["Documented overhead within agreed band","Mitigations merged","Residual fabric gap accepted or mitigated"],"risks":["Assuming matmul parity equals serving parity","Unpatched policy inversion","Incomplete attestation scope"]}},"retrievedAt":"2026-09-15T14:15:00Z","enrichedAt":"2026-09-15T14:17:00Z","enrichmentBasis":"retrieved source","accessibilityWorkforceImplications":"Performance taxes can shrink concurrent user capacity for campus confidential assistants unless mitigated.","procurementImplications":"Require workload-specific CC benchmarks and attestation scope in contracts; disallow sole reliance on GPU-local microbenchmarks.","operatingModelImplications":"Confidential AI platforms need ongoing runtime policy ownership, not one-time hardware enablement.","sourceVerification":{"openedUrl":"https://arxiv.org/html/2606.23969v2","referenceExcerpt":"LLM serving under Intel TDX plus GPU-CC still loses 13-27% of throughput","promptVersion":"sled-research-v4.0","model":null,"basis":"agent-reported inspection"}}}]}