# FastAFD: When Does Attention-FFN Disaggregation Pay on Rack-Scale NVLink?

DevFeed: [FastAFD: When Does Attention-FFN Disaggregation Pay on Rack-Scale NVLink?](<https://devfeed.tech/articles/fastafd-when-does-attention-ffn-disaggregation-pay-on-rack-scale-nvlink-120564.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.07149>)

Author: Yichao Fu, Yuxuan Zhang, Minshen Zhang, Ruitian Wang, Junda Chen, Hao Zhang

Published: 2026-11-06T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Distributed Computing](<https://devfeed.tech/sources/arxiv-distributed-computing.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Mixture of Experts (MoE)](<https://devfeed.tech/topics/mixture-of-experts-moe.md>), [vllm](<https://devfeed.tech/topics/vllm.md>), [GB200](<https://devfeed.tech/topics/gb200.md>), [NVLink](<https://devfeed.tech/topics/nvlink.md>), [groq](<https://devfeed.tech/topics/groq.md>)

Tags: [accelerators](<https://devfeed.tech/tags/accelerators.md>), [attention](<https://devfeed.tech/tags/attention.md>), [batch](<https://devfeed.tech/tags/batch.md>), [cost](<https://devfeed.tech/tags/cost.md>), [cs-dc](<https://devfeed.tech/tags/cs-dc.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [decode](<https://devfeed.tech/tags/decode.md>), [disaggregation](<https://devfeed.tech/tags/disaggregation.md>), [engine](<https://devfeed.tech/tags/engine.md>), [expert-parallelism](<https://devfeed.tech/tags/expert-parallelism.md>), [experts](<https://devfeed.tech/tags/experts.md>), [gb200](<https://devfeed.tech/tags/gb200.md>), [gpus](<https://devfeed.tech/tags/gpus.md>), [groq](<https://devfeed.tech/tags/groq.md>), [lpx](<https://devfeed.tech/tags/lpx.md>), [mixture-of-experts](<https://devfeed.tech/tags/mixture-of-experts.md>), [mixture-of-experts-moe](<https://devfeed.tech/tags/mixture-of-experts-moe.md>), [moe](<https://devfeed.tech/tags/moe.md>), [nvl72](<https://devfeed.tech/tags/nvl72.md>), [nvlink](<https://devfeed.tech/tags/nvlink.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [rack-scale](<https://devfeed.tech/tags/rack-scale.md>)

## AI overview

FastAFD separates attention from MoE experts on GB200 NVL72, overlapping one microbatch's MoE work with another's attention, and compares against vLLM's best wide expert-parallel setting. It reports up to 1.17x throughput on Qwen3, 1.16x on Qwen3.5, and parity on MiniMax-M2.5. Its model predicts AFD throughput within 3% from colocated vLLM profiles. The model attributes gains to larger batches and overlap, weighed against the GPUs allocated to experts; AFD pays off when the gains exceed that cost. Speculative decoding particularly helps light-expert models, adding 8% to AFD's advantage over wide EP. The abstract projects an ideal 2.48x speedup with speculative decoding if experts use on-chip SRAM accelerators, which would remove most disaggregation cost.

## Source excerpt

arXiv:2611.07149v1 Announce Type: cross Abstract: Attention-FFN disaggregation (AFD) separates attention from the experts of a mixture-of-experts (MoE) model and has been reported to raise decode throughput over colocated serving. Whether this advantage holds on rack-scale NVLink systems such as GB200 NVL72 remains unclear: prior results were obtained on other hardware, against narrower colocated baselines, or with analytical models, whereas production engines serve large MoE models on these systems with wide expert parallelism (wide EP). We build and tune FastAFD, an AFD engine for GB200 NVL72 that overlaps the MoE of one microbatch with the attention of the other, and compare it against the best vLLM wide EP setting. Experiments show that AFD has a throughput advantage of up to 1.17x over wide EP on Qwen3 and 1.16x on Qwen3.5, and matches it on MiniMax-M2.5. We derive an analytical model that predicts AFD throughput within 3% using only profiles of colocated vLLM, telling whether switching to AFD would pay off before deploying it. The model yields a concise insight: AFD's speedup factors into a batch gain (larger batches, since attention GPUs hold no expert weights), an overlap gain (shorter steps, since the MoE runs on the expert GPUs behind attention), and a disaggregation cost (the GPUs given to experts). AFD has the advantage when the two gains outweigh the cost. This view explains which models benefit from AFD, and shows that speculative decoding helps light-expert models most, where the enlarged overlap gain outweighs the larger disaggregation cost and adds a further 8% to AFD's advantage over wide EP. Looking ahead, serving the experts on accelerators that keep their weights in on-chip SRAM, such as the Groq LPX accelerators paired with Vera Rubin NVL72, would remove most of the disaggregation cost, raising the ideal speedup to 2.48x with speculative decoding. FastAFD is open source at https://github.com/hao-ai-lab/FastAFD.