# How Speculators and Mooncake enabled multi-node DSpark training for Kimi K3

DevFeed: [How Speculators and Mooncake enabled multi-node DSpark training for Kimi K3](<https://devfeed.tech/articles/how-we-trained-the-fastest-dspark-for-kimi-k3-using-gb300-nvl72-79487.md>)

Original publisher: [Read original article](<https://vllm.ai/blog/2026-09-15-kimi-k3-dspark>)

Author: Helen Zhao, Fynn Schmitt-Ulms, Yuchen Fama, Antonio J. Dominguez, and Kevin Li

Published: 2026-09-15T00:00:00Z

Content type: article

Language: en

Sources: [vLLM Blog](<https://devfeed.tech/sources/vllm-blog.md>)

Topics: [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Multi-GPU](<https://devfeed.tech/topics/multi-gpu.md>), [NCCL](<https://devfeed.tech/topics/nccl.md>), [GB300 NVL72](<https://devfeed.tech/topics/gb300-nvl72.md>), [NVFP4](<https://devfeed.tech/topics/nvfp4.md>)

Tags: [accelerators](<https://devfeed.tech/tags/accelerators.md>), [ai-engineering](<https://devfeed.tech/tags/ai-engineering.md>), [bare-metal](<https://devfeed.tech/tags/bare-metal.md>), [blackwell](<https://devfeed.tech/tags/blackwell.md>), [cuda](<https://devfeed.tech/tags/cuda.md>), [data-centers](<https://devfeed.tech/tags/data-centers.md>), [distributed](<https://devfeed.tech/tags/distributed.md>), [dspark](<https://devfeed.tech/tags/dspark.md>), [gb300](<https://devfeed.tech/tags/gb300.md>), [gb300-nvl72](<https://devfeed.tech/tags/gb300-nvl72.md>), [llm-inference](<https://devfeed.tech/tags/llm-inference.md>), [model-serving](<https://devfeed.tech/tags/model-serving.md>), [node](<https://devfeed.tech/tags/node.md>), [nvl72](<https://devfeed.tech/tags/nvl72.md>), [open-source-ai](<https://devfeed.tech/tags/open-source-ai.md>), [performance](<https://devfeed.tech/tags/performance.md>), [speculative-decoding](<https://devfeed.tech/tags/speculative-decoding.md>), [speculators](<https://devfeed.tech/tags/speculators.md>), [training](<https://devfeed.tech/tags/training.md>), [vllm-blog](<https://devfeed.tech/tags/vllm-blog.md>)

## AI overview

The vLLM Speculators library supports training and deploying DSpark speculative decoding models, with a Kimi K3 implementation reporting higher generation speed and throughput. A Mooncake connector streams hidden states between vLLM and training processes across nodes, enabling multi-node training for the 2.8-trillion-parameter model.

## Source excerpt

How Speculators and Mooncake enabled multi-node DSpark training for Kimi K3.