{"resourceId":"arxiv-fastafd-attention-ffn-disaggregation-gb200-nvl72-20261104","versions":[{"version":"external-bd2ac3a3007d64baa8ad68be829495c94701238fd53b993dca5a575a58aeda6c","resource":{"id":"arxiv-fastafd-attention-ffn-disaggregation-gb200-nvl72-20261104","title":"Preprint FastAFD: attention-FFN disaggregation gains up to 1.17x over the best wide-EP baseline on GB200 NVL72, and not on every model","organization":"Fu et al. (arXiv cs.DC)","sector":"Academic research / inference infrastructure","geography":"Not specified (authors' systems)","publishedAt":"2026-11-04","publicationDate":"2026-11-04","eventDate":null,"sourceName":"arXiv","sourceLabel":"arXiv","sourceUrl":"https://arxiv.org/abs/2611.07149","evidenceClass":"academic-research","outcomeClass":"mixed","topics":["infrastructure","developers-agents"],"finding":"The preprint builds FastAFD for GB200 NVL72 and compares attention-FFN disaggregation with the best vLLM wide expert-parallel setting. AFD gains up to 1.17x on Qwen3 and 1.16x on Qwen3.5 but only matches wide EP on MiniMax-M2.5. An analytical model predicts AFD throughput within 3% from colocated profiles, so operators can estimate payoff before deploying.","sledRelevance":"Mostly relevant to research universities and state research computing running MoE inference on rack-scale NVLink; a modest, workload-dependent gain rather than a general uplift. No SLED deployment is evaluated.","evidence":"arXiv:2611.07149v1 submitted Nov 4, 2026; abstract opened; experiments on GB200 NVL72 with Qwen3, Qwen3.5 and MiniMax-M2.5; code reported open source.","architectureImplications":"Serving design choice on rack-scale NVLink: dedicate GPUs to attention versus experts; benefit depends on batch gain, overlap gain and disaggregation cost.","governanceImplications":"Little governance content; relevant to benchmarking discipline: compare against the strongest baseline before adopting a new serving topology.","securityPrivacyImplications":"No security analysis; research code adopted from open source should follow normal supply-chain review.","caveats":"Preprint, not peer reviewed. Authors' systems and models; only abstract inspected here, full methods not independently reproduced; the 2.48x Groq LPX/Vera Rubin figure is a projection, not a measurement.","streamIds":["nvidia"],"roles":{"sales":"This preprint is an honest counterweight to blanket claims about disaggregated serving: the measured gain is up to 1.17x over a strong baseline and none on one model. Position GPU inference sizing around workload profiling rather than architecture hype. Offer research-computing teams a bounded inference-profiling engagement. Interpretation only: it evaluates model families on GB200 NVL72, not SLED workloads, so promised throughput or savings are unsupported. Preprint, not peer reviewed; the projected 2.48x on future accelerators is not measured evidence.","engineering":"The key engineering takeaway is that AFD pays off only when batch and overlap gains exceed the cost of GPUs given to experts, and the paper offers a model that predicts throughput within 3% from colocated vLLM profiles. Pre-sales should profile a customer's own MoE model before proposing disaggregation, and compare against tuned wide EP. Constraint: results are on GB200 NVL72 for three models. Interpretation: validate on the customer's model, context lengths and latency targets before committing to a topology.","delivery":"Delivery should treat topology as a measured decision: collect colocated vLLM profiles, apply the paper's model to estimate payoff, then run an A/B against tuned wide EP on the target rack. Dependencies include access to NVL72-class hardware, the open-source FastAFD code and representative traffic. Owner is the platform engineering lead. Interpretation: acceptance should be defined by the customer's own throughput and latency targets, not the paper's numbers, and preprint status means results may change."},"roleDetails":{"sales":{"customerProblem":"Research computing teams must decide how to serve large MoE models efficiently on rack-scale GPU systems without overbuying.","stakeholders":["Research computing director","Platform engineering lead","CIO","Principal investigators"],"discoveryQuestions":["Which MoE models do we serve and at what batch sizes?","Have we tuned wide expert parallelism first?","What latency target must we hold?"],"valueHypothesis":"Profiling-led topology selection could avoid unjustified architectural change and focus spend on measurable gains.","potentialEngagement":"Bounded inference profiling and topology comparison on one customer model.","unsupportedClaims":["Guaranteed throughput gain","That results transfer to SLED workloads","That the projected 2.48x speedup will materialize"]},"engineering":{"fit":"Relevant to MoE inference on GB200 NVL72-class racks.","architectureIntegration":"Compare AFD with tuned vLLM wide EP; AFD splits attention and expert GPUs.","prerequisites":["NVL72-class access","Colocated vLLM profiles","Representative traffic"],"constraints":["Three models tested","Gain up to 1.17x, parity on one model","Preprint"],"security":"Review open-source code supply chain before adoption.","proofOfValue":["Predict AFD payoff from profiles then measure against actual","A/B against tuned wide EP on customer model"]},"delivery":{"work":"Profile, model, and A/B test serving topology on the target rack.","dependencies":["Hardware access","Open-source FastAFD code","Representative workload traces"],"ownership":"Platform engineering lead.","skillsAdoption":"Train SREs on MoE serving metrics and profiling.","governanceCheckpoints":["Baseline sign-off","Result review before production change"],"acceptanceCriteria":["Predicted-versus-measured throughput gap reported","Adoption only if customer targets are met"],"risks":["Preprint results not replicating","Added operational complexity"]}},"retrievedAt":"2026-11-08T21:20:00-06:00","enrichedAt":"2026-11-08T21:50:00-06:00","enrichmentBasis":"retrieved source","accessibilityWorkforceImplications":"Limited direct workforce evidence in this source; interpretation is that operators need staff able to evaluate the claim before relying on it.","procurementImplications":"Interpretation: ask for primary evidence and measured acceptance criteria before using this source in a purchasing decision.","operatingModelImplications":"Interpretation: assign an operational owner to verify claims against the customer's own workloads or facilities.","peerReview":"preprint","sourceVerification":{"openedUrl":"https://arxiv.org/abs/2611.07149","referenceExcerpt":"Experiments show that AFD has a throughput advantage of up to 1.17x over wide EP on Qwen3 and 1.16x on Qwen3.5","promptVersion":"sled-research-v4.0","model":null,"basis":"agent-reported inspection"}}}]}