# flash-comm: Self-Synchronizing Collective Communication for LLM Decoding

DevFeed: [flash-comm: Self-Synchronizing Collective Communication for LLM Decoding](<https://devfeed.tech/articles/flash-comm-self-synchronizing-collective-communication-for-llm-decoding-119117.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.06634>)

Author: Carrie Wang, Jack Li, Jason Zhao

Published: 2026-11-05T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Distributed Computing](<https://devfeed.tech/sources/arxiv-distributed-computing.md>)

Topics: [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>), [NCCL](<https://devfeed.tech/topics/nccl.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>)

Tags: [asynchronous](<https://devfeed.tech/tags/asynchronous.md>), [b300](<https://devfeed.tech/tags/b300.md>), [collective](<https://devfeed.tech/tags/collective.md>), [communication](<https://devfeed.tech/tags/communication.md>), [consistency](<https://devfeed.tech/tags/consistency.md>), [cost](<https://devfeed.tech/tags/cost.md>), [cs-dc](<https://devfeed.tech/tags/cs-dc.md>), [data-movement](<https://devfeed.tech/tags/data-movement.md>), [decoding](<https://devfeed.tech/tags/decoding.md>), [encoding](<https://devfeed.tech/tags/encoding.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [gpus](<https://devfeed.tech/tags/gpus.md>), [latency](<https://devfeed.tech/tags/latency.md>), [library](<https://devfeed.tech/tags/library.md>), [llm](<https://devfeed.tech/tags/llm.md>), [median](<https://devfeed.tech/tags/median.md>), [message](<https://devfeed.tech/tags/message.md>), [protocol](<https://devfeed.tech/tags/protocol.md>), [reduction](<https://devfeed.tech/tags/reduction.md>), [selector](<https://devfeed.tech/tags/selector.md>), [synchronization](<https://devfeed.tech/tags/synchronization.md>)

## AI overview

flash-comm fuses readiness synchronization with data movement: each data block carries readiness evidence so receivers can validate needed blocks without separate inter-GPU barriers. A cost-guided selector adapts readiness encoding to message size and GPU count, balancing synchronization overhead against extra transfer cost. It also separates transport order from reduction order to preserve result consistency with asynchronous arrivals. On eight NVIDIA B300 GPUs, all-reduce latency fell by a median 28.8% and up to 43.7% versus custom AR v2; end-to-end decoding time per output token fell 1.7-6.8%. Since all-reduce was 11.4% of baseline decoding time, optimizing it alone limits possible overall latency gains. Standalone all-gather latency fell by a median 23.8% and up to 58.9%.

## Source excerpt

arXiv:2611.06634v1 Announce Type: new Abstract: We present flash-comm, a communication library that achieves efficient transport for LLM decoding by fusing readiness synchronization with data movement. flash-comm introduces self-synchronizing transfers in which each data block carries evidence of its readiness, allowing receivers to directly validate the blocks they need without separate inter-GPU barriers. Based on this principle, a cost-guided protocol selector adapts the readiness encoding to message size and the number of participating GPUs, balancing synchronization overhead against additional data-transfer cost. flash-comm further decouples transport order from reduction order to preserve result consistency across GPUs under asynchronous data arrivals. On a server with eight NVIDIA B300 GPUs, flash-comm reduces all-reduce latency by a median of 28.8% and up to 43.7% compared with custom AR v2 (the state-of-the-art all-reduce implementation). In end-to-end decoding, flash-comm reduces time per output token (TPOT) by 1.7-6.8%. Note that all-reduce accounts for 11.4% of the baseline decoding time, which bounds the latency reduction possible from optimizing this operation alone. The same mechanism also supports standalone all-gather, reducing latency by a median of 23.8% and up to 58.9% across eight B300 GPUs.