Lighthouse AdvisorySLED AI Adoption Intelligence
← Back to results

From the NVIDIA edition of November 8, 2026

Academic researchMixedPreprint, not peer reviewedNew this fortnight

Preprint FastAFD: attention-FFN disaggregation gains up to 1.17x over the best wide-EP baseline on GB200 NVL72, and not on every model

Fu et al. (arXiv cs.DC) · Academic research / inference infrastructure · Not specified (authors' systems)

NVIDIA
Publisher
arXiv
Original publication
2026-11-04
Source retrieved
2026-11-09
Read original source

What happened

The preprint builds FastAFD for GB200 NVL72 and compares attention-FFN disaggregation with the best vLLM wide expert-parallel setting. AFD gains up to 1.17x on Qwen3 and 1.16x on Qwen3.5 but only matches wide EP on MiniMax-M2.5. An analytical model predicts AFD throughput within 3% from colocated profiles, so operators can estimate payoff before deploying.

Why it matters

Mostly relevant to research universities and state research computing running MoE inference on rack-scale NVLink; a modest, workload-dependent gain rather than a general uplift. No SLED deployment is evaluated.

Evidence and measured results

arXiv:2611.07149v1 submitted Nov 4, 2026; abstract opened; experiments on GB200 NVL72 with Qwen3, Qwen3.5 and MiniMax-M2.5; code reported open source.

Limitations and uncertainty

Preprint, not peer reviewed. Authors' systems and models; only abstract inspected here, full methods not independently reproduced; the 2.48x Groq LPX/Vera Rubin figure is a projection, not a measurement.

Cross-source patterns this evidence supports

Put this evidence to work

Lighthouse Advisory interpretation, grounded in this source. Enriched 2026-11-08; this does not change the original publication date. Each role follows the same checklist so takeaways compare across sources.

Sales

Role takeaway
Customer problem
Research computing teams must decide how to serve large MoE models efficiently on rack-scale GPU systems without overbuying.
Stakeholders
  • Research computing director
  • Platform engineering lead
  • CIO
  • Principal investigators
Discovery questions
  • Which MoE models do we serve and at what batch sizes?
  • Have we tuned wide expert parallelism first?
  • What latency target must we hold?
Value hypothesis
Profiling-led topology selection could avoid unjustified architectural change and focus spend on measurable gains.
Potential engagement
Bounded inference profiling and topology comparison on one customer model.
Unsupported claims
  • Guaranteed throughput gain
  • That results transfer to SLED workloads
  • That the projected 2.48x speedup will materialize

Pre-sales engineering

Role takeaway
Fit
Relevant to MoE inference on GB200 NVL72-class racks.
Architecture and integration
Compare AFD with tuned vLLM wide EP; AFD splits attention and expert GPUs.
Prerequisites
  • NVL72-class access
  • Colocated vLLM profiles
  • Representative traffic
Constraints
  • Three models tested
  • Gain up to 1.17x, parity on one model
  • Preprint
Security
Review open-source code supply chain before adoption.
Proof of value
  • Predict AFD payoff from profiles then measure against actual
  • A/B against tuned wide EP on customer model

Delivery

Role takeaway
Work
Profile, model, and A/B test serving topology on the target rack.
Dependencies
  • Hardware access
  • Open-source FastAFD code
  • Representative workload traces
Ownership
Platform engineering lead.
Skills and adoption
Train SREs on MoE serving metrics and profiling.
Governance checkpoints
  • Baseline sign-off
  • Result review before production change
Acceptance criteria
  • Predicted-versus-measured throughput gap reported
  • Adoption only if customer targets are met
Risks
  • Preprint results not replicating
  • Added operational complexity

Implementation considerations

Lighthouse Advisory interpretation across the operating dimensions a public-sector buyer must settle before this evidence becomes a design. Each note answers the question under its heading for this specific source.

Architecture and integration

What must connect, and where does the AI sit in the workflow?

Serving design choice on rack-scale NVLink: dedicate GPUs to attention versus experts; benefit depends on batch gain, overlap gain and disaggregation cost.

Governance

Who approves, reviews and stays accountable for outcomes?

Little governance content; relevant to benchmarking discipline: compare against the strongest baseline before adopting a new serving topology.

Security and privacy

What data, permissions and controls need testing?

No security analysis; research code adopted from open source should follow normal supply-chain review.

Accessibility and workforce

Who is affected, and what skills or accommodations follow?

Limited direct workforce evidence in this source; interpretation is that operators need staff able to evaluate the claim before relying on it.

Procurement

What should contracts, pricing and exit terms secure?

Ask for primary evidence and measured acceptance criteria before using this source in a purchasing decision.

Operating model

Which teams own the service once it runs?

Assign an operational owner to verify claims against the customer's own workloads or facilities.

Publication history

  1. 2026-11-08NVIDIA · Issue 646 resources
Read preserved resource versions (JSON)

Stable resource ID: arxiv-fastafd-attention-ffn-disaggregation-gb200-nvl72-20261104