# Setting a World Record for MoE Pre-Training on NVIDIA GB300 NVL72

DevFeed: [Setting a World Record for MoE Pre-Training on NVIDIA GB300 NVL72](<https://devfeed.tech/articles/setting-a-world-record-for-moe-pre-training-on-nvidia-gb300-nvl72-6939.md>)

Original publisher: [Read original article](<https://developer.nvidia.com/blog/setting-a-world-record-for-moe-pre-training-on-nvidia-gb300-nvl72/>)

Author: Kirthi Devleker

Published: 2026-07-21T18:30:00Z

Content type: article

Language: en

Sources: [NVIDIA Developer](<https://devfeed.tech/sources/nvidia-developer.md>), [NVIDIA Technical Blog](<https://devfeed.tech/sources/nvidia-technical-blog.md>)

Topics: [Mixture of Experts (MoE)](<https://devfeed.tech/topics/mixture-of-experts-moe.md>), [GPU](<https://devfeed.tech/topics/gpu.md>), [Nvidia](<https://devfeed.tech/topics/nvidia.md>), [AI Platform](<https://devfeed.tech/topics/ai-platform.md>), [NCCL](<https://devfeed.tech/topics/nccl.md>), [networking](<https://devfeed.tech/topics/networking.md>), [deepseek](<https://devfeed.tech/topics/deepseek.md>)

Tags: [agentic-ai-generative-ai](<https://devfeed.tech/tags/agentic-ai-generative-ai.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-platform](<https://devfeed.tech/tags/ai-platform.md>), [ai-training](<https://devfeed.tech/tags/ai-training.md>), [collective](<https://devfeed.tech/tags/collective.md>), [communication](<https://devfeed.tech/tags/communication.md>), [compute](<https://devfeed.tech/tags/compute.md>), [data-center-cloud](<https://devfeed.tech/tags/data-center-cloud.md>), [deepseek](<https://devfeed.tech/tags/deepseek.md>), [developer-tools-techniques](<https://devfeed.tech/tags/developer-tools-techniques.md>), [efficiency](<https://devfeed.tech/tags/efficiency.md>), [featured](<https://devfeed.tech/tags/featured.md>), [frontier-model](<https://devfeed.tech/tags/frontier-model.md>), [gb300-nvl72](<https://devfeed.tech/tags/gb300-nvl72.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [infrastructure](<https://devfeed.tech/tags/infrastructure.md>), [latency](<https://devfeed.tech/tags/latency.md>), [llm-techniques](<https://devfeed.tech/tags/llm-techniques.md>), [low-latency](<https://devfeed.tech/tags/low-latency.md>), [megatron](<https://devfeed.tech/tags/megatron.md>), [mixture-of-experts-moe](<https://devfeed.tech/tags/mixture-of-experts-moe.md>), [moe](<https://devfeed.tech/tags/moe.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [nvl72](<https://devfeed.tech/tags/nvl72.md>), [performance](<https://devfeed.tech/tags/performance.md>), [top-stories](<https://devfeed.tech/tags/top-stories.md>), [train](<https://devfeed.tech/tags/train.md>), [training-ai-models](<https://devfeed.tech/tags/training-ai-models.md>)

## AI overview

The article explains how NVIDIA GB300 NVL72 achieved a world record for DeepSeek-V3 671B mixture-of-experts pre-training. It focuses on the communication demands of MoE models, including all-to-all traffic between GPUs, and the need for tightly coupled scale-up and predictable scale-out networking to sustain delivered training performance.

## Source excerpt

Frontier model pre-training has converged on mixture of experts (MoE), which is fundamentally changing what limits large-scale AI training. As compute per token...