# Megatron

Published articles for Megatron.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## Developing Nemotron 3.5 Lightning NVFP4 with QAD Using NVIDIA Model Optimizer

DevFeed: [Developing Nemotron 3.5 Lightning NVFP4 with QAD Using NVIDIA Model Optimizer](<https://devfeed.tech/articles/developing-nemotron-3-5-lightning-nvfp4-with-qad-using-nvidia-model-optimizer-6811.md>)

Original publisher: [Read original article](<https://developer.nvidia.com/blog/developing-nemotron-3-5-lightning-nvfp4-with-qad-using-nvidia-model-optimizer/>)

Author: Tanya Lenz

Published: 2026-08-17T18:12:48Z

Content type: tutorial

Language: en

Sources: [NVIDIA Developer](<https://devfeed.tech/sources/nvidia-developer.md>), [NVIDIA Technical Blog](<https://devfeed.tech/sources/nvidia-technical-blog.md>)

Topics: [Nemotron](<https://devfeed.tech/topics/nemotron.md>), [NVFP4](<https://devfeed.tech/topics/nvfp4.md>), [quantization](<https://devfeed.tech/topics/quantization.md>), [Post-training optimization](<https://devfeed.tech/topics/post-training-optimization.md>), [Nvidia](<https://devfeed.tech/topics/nvidia.md>), [post-training](<https://devfeed.tech/topics/post-training.md>), [Mamba](<https://devfeed.tech/topics/mamba.md>)

Tags: [agentic-ai-generative-ai](<https://devfeed.tech/tags/agentic-ai-generative-ai.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [compute](<https://devfeed.tech/tags/compute.md>), [developer-tools-techniques](<https://devfeed.tech/tags/developer-tools-techniques.md>), [developers](<https://devfeed.tech/tags/developers.md>), [edge-computing](<https://devfeed.tech/tags/edge-computing.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [featured](<https://devfeed.tech/tags/featured.md>), [latency](<https://devfeed.tech/tags/latency.md>), [mamba](<https://devfeed.tech/tags/mamba.md>), [megatron](<https://devfeed.tech/tags/megatron.md>), [memory](<https://devfeed.tech/tags/memory.md>), [model](<https://devfeed.tech/tags/model.md>), [model-optimizer](<https://devfeed.tech/tags/model-optimizer.md>), [models](<https://devfeed.tech/tags/models.md>), [nemotron](<https://devfeed.tech/tags/nemotron.md>), [nvfp4](<https://devfeed.tech/tags/nvfp4.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [open](<https://devfeed.tech/tags/open.md>), [post-training](<https://devfeed.tech/tags/post-training.md>), [quantization](<https://devfeed.tech/tags/quantization.md>), [speed](<https://devfeed.tech/tags/speed.md>), [training](<https://devfeed.tech/tags/training.md>), [training-ai-models](<https://devfeed.tech/tags/training-ai-models.md>)

### AI overview

This tutorial explains how quantization-aware distillation (QAD) creates the Nemotron 3.5 Lightning NVFP4 checkpoint using NVIDIA Model Optimizer. It covers post-training quantization, teacher-student distillation, and evaluation, showing how QAD can recover accuracy while reducing memory usage and increasing throughput.

### Source excerpt

Teams customize their models to hit their targets for latency, speed, memory, and compute. With the open NVIDIA Nemotron family of models, developers can find...

## Setting a World Record for MoE Pre-Training on NVIDIA GB300 NVL72

DevFeed: [Setting a World Record for MoE Pre-Training on NVIDIA GB300 NVL72](<https://devfeed.tech/articles/setting-a-world-record-for-moe-pre-training-on-nvidia-gb300-nvl72-6939.md>)

Original publisher: [Read original article](<https://developer.nvidia.com/blog/setting-a-world-record-for-moe-pre-training-on-nvidia-gb300-nvl72/>)

Author: Kirthi Devleker

Published: 2026-07-21T18:30:00Z

Content type: article

Language: en

Sources: [NVIDIA Developer](<https://devfeed.tech/sources/nvidia-developer.md>), [NVIDIA Technical Blog](<https://devfeed.tech/sources/nvidia-technical-blog.md>)

Topics: [Mixture of Experts (MoE)](<https://devfeed.tech/topics/mixture-of-experts-moe.md>), [GPU](<https://devfeed.tech/topics/gpu.md>), [Nvidia](<https://devfeed.tech/topics/nvidia.md>), [AI Platform](<https://devfeed.tech/topics/ai-platform.md>), [NCCL](<https://devfeed.tech/topics/nccl.md>), [networking](<https://devfeed.tech/topics/networking.md>), [deepseek](<https://devfeed.tech/topics/deepseek.md>)

Tags: [agentic-ai-generative-ai](<https://devfeed.tech/tags/agentic-ai-generative-ai.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-platform](<https://devfeed.tech/tags/ai-platform.md>), [ai-training](<https://devfeed.tech/tags/ai-training.md>), [collective](<https://devfeed.tech/tags/collective.md>), [communication](<https://devfeed.tech/tags/communication.md>), [compute](<https://devfeed.tech/tags/compute.md>), [data-center-cloud](<https://devfeed.tech/tags/data-center-cloud.md>), [deepseek](<https://devfeed.tech/tags/deepseek.md>), [developer-tools-techniques](<https://devfeed.tech/tags/developer-tools-techniques.md>), [efficiency](<https://devfeed.tech/tags/efficiency.md>), [featured](<https://devfeed.tech/tags/featured.md>), [frontier-model](<https://devfeed.tech/tags/frontier-model.md>), [gb300-nvl72](<https://devfeed.tech/tags/gb300-nvl72.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [infrastructure](<https://devfeed.tech/tags/infrastructure.md>), [latency](<https://devfeed.tech/tags/latency.md>), [llm-techniques](<https://devfeed.tech/tags/llm-techniques.md>), [low-latency](<https://devfeed.tech/tags/low-latency.md>), [megatron](<https://devfeed.tech/tags/megatron.md>), [mixture-of-experts-moe](<https://devfeed.tech/tags/mixture-of-experts-moe.md>), [moe](<https://devfeed.tech/tags/moe.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [nvl72](<https://devfeed.tech/tags/nvl72.md>), [performance](<https://devfeed.tech/tags/performance.md>), [top-stories](<https://devfeed.tech/tags/top-stories.md>), [train](<https://devfeed.tech/tags/train.md>), [training-ai-models](<https://devfeed.tech/tags/training-ai-models.md>)

### AI overview

The article explains how NVIDIA GB300 NVL72 achieved a world record for DeepSeek-V3 671B mixture-of-experts pre-training. It focuses on the communication demands of MoE models, including all-to-all traffic between GPUs, and the need for tightly coupled scale-up and predictable scale-out networking to sustain delivered training performance.

### Source excerpt

Frontier model pre-training has converged on mixture of experts (MoE), which is fundamentally changing what limits large-scale AI training. As compute per token...