# H100 Published articles for H100. This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## DMuon: Scaling Matrix Optimizers in Data Parallelism with Topology-Aware Matrix Ownership and Communication DevFeed: [DMuon: Scaling Matrix Optimizers in Data Parallelism with Topology-Aware Matrix Ownership and Communication]() Original publisher: [Read original article]() Author: Starrick Liu, Ruizhi Chengze, Lufang Chen, Michael Zhao, Ran Yu, Xiaofan Li, dance kuzi, Yue Zhang, Zijuan Lin, SongAo Han, YongHua Chen, Ruyi Gan, Hao Wang Published: 2026-11-06T05:00:00Z Content type: article Language: en Sources: [arXiv: Distributed Computing]() Topics: [distributed-training](), [Distributed Systems & Parallel Computing](), [sharding](), [High-Performance Computing]() Tags: [cluster](), [collective](), [communication](), [cost](), [cs-dc](), [gathering](), [global](), [gpus](), [gradient](), [h100](), [iteration](), [layout](), [llms](), [load](), [matrix](), [muon](), [network](), [ownership](), [parallelism](), [scaling](), [topology]() ### AI overview DMuon makes matrix optimizers practical with sharded data-parallel training by assigning each weight matrix to one owner, which runs the Newton-Schulz iteration and update on the full matrix. Owners are balanced by computational cost. In HSDP, ownership spans replicas, keeping one cluster-wide copy of gradients and optimizer state; updated parameters are published across nodes as they become ready. DMuon schedules sparse, variable-length collectives as data becomes available and routes them over the cluster's two-level network. Across three dense LLMs and two VLA models on up to 256 H100 GPUs, it matched or exceeded PyTorch FSDP2 with AdamW, reaching 1.05x its median throughput at 256 GPUs. It achieved 2.1x the median throughput of a gather-then-replicate Muon baseline. These are reported results for the tested models and scale. ### Source excerpt arXiv:2611.07084v1 Announce Type: new Abstract: Matrix optimizers such as Muon update whole weight matrices, which conflicts with the element- or row-wise sharding of data-parallel runtimes. Once parameters are sharded as well, matrix optimizers incur extra communication or repeated computation. We present DMuon, a matrix-granular optimizer runtime for PyTorch FSDP (ZeRO-2/3) and HSDP. DMuon assigns each matrix a unique owner, which performs the Newton--Schulz (NS) iteration and optimizer update on the full matrix, and balances the load across owners by computational cost. On HSDP, ownership is global across replicas, so the cluster keeps a single copy of gradients and optimizer state, and updated parameters are published across nodes as soon as they are ready, overlapping subsequent computation. This layout makes every collective sparse and variable in length, so DMuon schedules these collectives as their data becomes ready and routes them along the cluster's two-level network. On up to 256 H100 GPUs, across three dense LLMs and two VLA models, DMuon matches or exceeds the training throughput of PyTorch FSDP2 with AdamW, reaching 1.05$\times$ that of FSDP2 with AdamW at the median on 256 GPUs. The optimized parameter gathering and gradient reduction absorb the extra communication and computation a matrix optimizer requires. Against a gather-then-replicate Muon baseline, DMuon's throughput is 2.1 times as high at the median. ## AutoCompress: Automated Hardware-Guided Model Compression DevFeed: [AutoCompress: Automated Hardware-Guided Model Compression]() Original publisher: [Read original article]() Author: Sheldon Liang, Ali Taghibakhshi, Saurav Muralidharan, Sharath Turuvekere Sreenivas, Ashwath Aithal, Pavlo Molchanov, Nima Tajbakhsh Published: 2026-11-06T05:00:00Z Content type: article Language: en Sources: [arXiv: Artificial Intelligence](), [arXiv: Machine Learning]() Topics: [Post-training optimization](), [LLM optimization](), [model architecture](), [Large Language Model](), [performance-engineering]() Tags: [architecture](), [benchmark](), [coding](), [compression](), [context](), [cs-ai](), [cs-lg](), [datacenter](), [engine](), [framework](), [gpu](), [gpu-memory](), [h100](), [hardware](), [interaction](), [latency](), [layer](), [llm](), [long-context](), [measurements](), [memory](), [model-compression]() ### AI overview AutoCompress searches for model structures that satisfy hardware- and workload-specific serving limits, rather than relying on parameter count alone. It jointly optimizes global and per-layer structure to create heterogeneous architectures, using serving-engine cost models validated against measured latency, throughput, and memory. Across long-context local coding, batched datacenter serving, and latency-sensitive in-game interaction, it improved average accuracy over Minitron by 6.57, 2.35, and 0.57 percentage points on RTX 5070, RTX 5080, and H100, respectively, across eight benchmarks. In another result, it compressed Nemotron 3.5 Super from 120B to 74B parameters for single-B200 BF16 serving at 128K context while retaining 95% of its mean benchmark score. ### Source excerpt arXiv:2611.08290v1 Announce Type: cross Abstract: LLM pruning must preserve quality while meeting hardware- and workload-specific constraints on latency, throughput, and memory. Existing pruning workflows often target parameter counts, requiring practitioners to manually select model sizes that meet latency, throughput, and GPU memory limits. However, parameter count alone does not predict these metrics, which depend on the model architecture, workload, hardware, and serving engine. We introduce AutoCompress, a hardware-guided compression framework that directly incorporates such constraints into differentiable architecture search. AutoCompress jointly optimizes global and per-layer structure to produce heterogeneous architectures, guided by serving-engine cost models validated against end-to-end latency, throughput, and memory measurements. We evaluate AutoCompress on long-context local coding, batched datacenter serving, and latency-sensitive in-game interaction. Compared to the state-of-the-art Minitron pruning framework, AutoCompress improves average accuracy across eight benchmarks by 6.57, 2.35, and 0.57 percentage points on RTX 5070, RTX 5080, and H100, respectively. AutoCompress also compresses Nemotron 3.5 Super from 120B to 74B parameters for single-B200 BF16 serving at 128K context, retaining 95% of its mean benchmark score. ## Five Ways NVIDIA Co-Design Maximizes the Value of Every Byte in the AI Factory DevFeed: [Five Ways NVIDIA Co-Design Maximizes the Value of Every Byte in the AI Factory]() Original publisher: [Read original article]() Author: Shara Tibken Published: 2026-11-05T16:00:00Z Content type: article Language: en Sources: [NVIDIA Developer](), [NVIDIA Technical Blog]() Topics: [AI Factory](), [AI Architecture](), [NVFP4](), [NVLink](), [Multi-GPU](), [vllm]() Tags: [agentic](), [agentic-ai-generative-ai](), [agentic-workloads](), [agents](), [ai-factory](), [architecture](), [arithmetic](), [attention](), [batch](), [benchmarks](), [better](), [bits](), [block](), [bluefield-dpu](), [cache](), [cached-context](), [capacity](), [co-design](), [data-center-cloud](), [doca](), [dynamo](), [ethernet](), [featured](), [gb200](), [gb300](), [gb300-nvl72](), [gpudirect](), [h100](), [hbm](), [high-bandwidth-memory](), [memory](), [nvidia](), [nvlink](), [rubin-gpu](), [spectrum-x](), [tensorrt-llm](), [vera-cpu](), [vera-rubin](), [vera-rubin-nvl72]() ### AI overview The excerpt describes five ways NVIDIA's AI factory design aims to increase useful work per memory byte: NVFP4 shrinks model weights and KV cache; TensorRT-LLM Wide-EP spreads MoE experts across GPUs to free per-GPU memory and improve weight reuse; Dynamo routes requests to cached prefixes and manages KV cache across GPU, CPU, and storage; Vera CPU memory provides a place to stage context during tool-call gaps; and CMX offers pod-level shared flash for context reuse across nodes. The article reports benchmark-specific gains, including up to 4x throughput over an FP8 baseline for NVFP4 and up to 30x throughput per megawatt for Vera Rubin NVL72 versus GB300 NVL72 at the same 288 GB per GPU. It also notes that KV offloading helps when retrieval is faster than recomputation; storage performance examples are vendor validation tests. ### Source excerpt In an AI factory, high bandwidth memory (HBM)is the fastest, most expensive, and most contested memory tier. Every byte spent on a redundant weight copy, an... ## Energy-Efficient Reinforcement Learning for Large Language Models DevFeed: [Energy-Efficient Reinforcement Learning for Large Language Models]() Original publisher: [Read original article]() Author: Rahul Sharma, Nitin Vegesna, Yanxin Zhang Published: 2026-11-05T05:00:00Z Content type: article Language: en Sources: [arXiv: Distributed Computing]() Topics: [Reinforcement learning](), [LLM optimization](), [LLM Techniques](), [Large Language Model]() Tags: [asynchronous](), [controller](), [cs-dc](), [energy](), [gb200](), [gpu](), [h100](), [language-models](), [online](), [reinforcement-learning](), [throughput](), [training]() ### AI overview EE-RL reduces GPU energy during language-model reinforcement learning by choosing separate GPU frequencies for response generation and policy training. It compares candidate settings against a prediction of the default GPU governor on the same token workload, selects the most energy-efficient setting that meets a throughput threshold, then monitors and adjusts as workloads change. Fixed-frequency sweeps on H100 and GB200 found savings of up to 28.2% GPU energy per valid training token across synchronous and asynchronous execution while largely preserving throughput. In online operation, EE-RL reduced energy per valid training token by up to 16.9%, at a modest throughput cost. ### Source excerpt arXiv:2611.05748v1 Announce Type: new Abstract: Reinforcement learning for language models alternates between generating responses and training the policy. These stages place different demands on GPUs, creating opportunities to reduce clock frequency with little loss in throughput. Selecting frequencies online is difficult because response lengths change and frequency trials themselves consume time and energy. We present EE-RL (energy-efficient RL), an online controller that selects separate frequencies for generation and training during RL execution. It compares candidate settings with a prediction of the default GPU governor on the same token workload, and keeps the most energy-efficient setting that meets a throughput threshold. The controller then monitors throughput and energy and adjusts frequencies as the workload changes. Fixed-frequency sweeps on H100 and GB200 show GPU energy savings of up to 28.2% per valid training token across synchronous and asynchronous execution, while largely preserving throughput. EE-RL turns this headroom into online savings, reducing GPU energy per valid training token by up to 16.9% at a modest throughput cost. ## vExpert: Virtualizing Expert Storage for Adaptive Load Balancing in Distributed MoE Inference DevFeed: [vExpert: Virtualizing Expert Storage for Adaptive Load Balancing in Distributed MoE Inference]() Original publisher: [Read original article]() Author: Wenxun Wang, Xiuhong Li, Yida Wang, Chen Tang, Zongle Huang, Ke Hong, Yu Wang, Yongpan Liu Published: 2026-11-05T05:00:00Z Content type: article Language: en Sources: [arXiv: Distributed Computing]() Topics: [Mixture of Experts (MoE)](), [Multi-GPU](), [Load Balancing]() Tags: [adaptive](), [balancing](), [cs-dc](), [distributed](), [distributed-inference](), [dram](), [h100](), [h100-cluster](), [inference](), [latency](), [load-balancing](), [memory](), [moe](), [offloading](), [overhead](), [pcie](), [performance](), [ratio](), [routing](), [runtime](), [storage]() ### AI overview vExpert addresses dynamic load imbalance in distributed MoE inference by using host DRAM for full redundancy of virtualized expert storage. Its performance model jointly accounts for host-to-device transfer latency, shared PCIe contention, and load imbalance; a lightweight allocator maps physical experts to virtual slots at runtime, while a disaggregated manager reduces memory overhead across processes. In an evaluation on DeepSeek-V3.1 using an 8-node H100 cluster, the authors report up to 55% improvement in load balance ratio and 1.2x end-to-end speedup. The supplied text is an abstract, so it does not provide further evaluation details. ### Source excerpt arXiv:2611.06057v1 Announce Type: new Abstract: Mixture-of-Experts (MoE) models have become prevailing, yet their distributed deployment suffers from severe dynamic load imbalance due to the conflict between static expert placement and inherent routing dynamism. While host DRAM offers capacity to avoid static device storage for load balancing, we identify existing offloading works fundamentally fail in distributed inference due to two overlooked challenges: (1) coupled performance impacts of runtime Host-to-Device transfers due to load dynamism and shared PCIe topology; (2) system-level memory overhead incurred by DRAM usage at scale. In this paper, we propose vExpert that shifts DRAM offloading paradigm from partial residency to full redundancy through virtualized expert storage. We pioneer the first systematic performance model that jointly captures H2D latency, PCIe contention, and load imbalance, building a lightweight allocator to dynamically map physical experts to virtual slots runtime. A novel disaggregated manager is designed to reduce memory overhead in multi-process environments. Evaluated on DeepSeek-V3.1 within an 8-node H100 cluster, vExpert improves load balance ratio by up to 55% and achieves 1.2$\times$ end-to-end speedup. ## Enforcing Accountable Disclosure in Confidential AI DevFeed: [Enforcing Accountable Disclosure in Confidential AI]() Original publisher: [Read original article]() Author: Shumpei Shiina, Haruka Takahashi, Kento Sasaki, Takuya Hayashi, Tsubasa Takahashi Published: 2026-11-05T05:00:00Z Content type: article Language: en Sources: [arXiv: Cryptography and Security]() Topics: [Confidential Computing](), [ai-governance](), [Data Management](), [virtual machines]() Tags: [agent](), [ai](), [attestation](), [confidential-computing](), [cs-cr](), [cs-os](), [data](), [data-management](), [derived-data](), [disclosure](), [external-services](), [gpu](), [h100](), [information-flow-control](), [latency](), [ledger](), [rust](), [safety](), [system-design](), [tee](), [transparent](), [virtual-machines]() ### AI overview CORDE applies a "Confine or Declare" rule to confidential AI: user-derived data stays within a confidential virtual machine by default, and disclosure requires recording the exact data, recipient, and stated reason in a user-accessible, tamper-evident log. Its design combines per-user information-flow control in Rust with a TEE-based ledger and remote attestation. On an NVIDIA H100 GPU, the authors report that declaration before disclosure added about 0.5% task latency on the AppWorld agent benchmark. The supplied text is an abstract and does not provide further implementation or evaluation details. ### Source excerpt arXiv:2611.05971v1 Announce Type: new Abstract: Advances in CPU and GPU confidential computing enable confidential AI, allowing LLMs to process user prompts without exposing them to service providers or infrastructure operators. However, AI agents often need to send prompt-derived information to external services, and providers may also require limited disclosure for safety monitoring and failure diagnosis. We propose CORDE, a system for confidential virtual machines (CVMs) designed to satisfy the principle of Confine or Declare: user-derived data is confined by default, but when necessary, it may leave the CVM only after being "declared." To "declare" data is to record its exact contents, recipient, and stated reason in a user-accessible, tamper-evident declaration log. To put this principle into practice, we present an integrated system design that combines per-user information-flow control in Rust with a TEE-based ledger and remote attestation. Our evaluation on an NVIDIA H100 GPU shows that declaration before disclosure adds little overhead in task latency, about 0.5% on the AppWorld agent benchmark. CORDE provides a basis for accountable disclosure of user data, supporting responsible and transparent data management in AI services. ## Runway Characters: Serving a 57B Video Diffusion Transformer for Real-Time Conversation DevFeed: [Runway Characters: Serving a 57B Video Diffusion Transformer for Real-Time Conversation]() Original publisher: [Read original article]() Author: Rohan Agarwal, Robin Andeer, Alex Armbruster, Bennett Arthur, Michail Christos Doukas, Piero Esposito, Qiming Fang, Aron Filkey, Bryan Fox, Anastasis Germanidis, Jonathan Granskog, Corina Gurau, Robin Published: 2026-11-05T05:00:00Z Content type: article Language: en Sources: [arXiv: Distributed Computing]() Topics: [diffusion-transformers](), [Real-time AI interactions](), [real-time]() Tags: [attention](), [autoregressive](), [cache](), [cs-dc](), [decoder](), [denoising](), [diffusion](), [fp8](), [gpu](), [h100](), [kernel](), [kv-cache](), [latency](), [open-source](), [optimizations](), [production](), [real-time](), [runway](), [serving](), [transformer](), [video]() ### AI overview Runway Characters serves an approximately 57B-parameter autoregressive video diffusion model at 704x1088 and 24 fps on four H100 GPUs, with a voice agent in the loop. To reduce repeated work across 240 attention calls per four-frame chunk, the system moves KV-cache processing to write time using a ring cache, one-time RoPE and FP8 quantization, and a delta rotation when positions reset. It also fuses tensor-parallel all-reduce into the FP8 GEMM using in-switch reduction and assigns video-frame decoding its own GPU. The diffusion step fell from 240.9 ms to 159.3 ms without the fused kernel, with no detected lip-sync-score degradation; the fused version reaches 147.4 ms, after a production rollback and re-enablement. Over seven production weeks, 99.5% of sessions streaming at least one second averaged above 23.5 fps. The abstract says two open-source systems tested on the same GPUs at four denoising passes missed real time or added over a second of latency, at one-third to one-half the resolution; one latency figure is an estimate. ### Source excerpt arXiv:2611.05642v1 Announce Type: new Abstract: Conversational video characters must generate video as fast as it plays, for one user at a time: at 24 fps, a sustained 166.7 ms per four-frame chunk at batch size one, with only a few chunks of buffering. We describe the system behind Runway Characters, which serves a $\approx$57B-parameter autoregressive video diffusion transformer (four denoising passes per chunk) at 704$\times$1088 and 24 fps on four H100 GPUs with a voice agent in the loop. Each chunk takes 240 attention calls, each reading a 5,788-slot KV cache in which only 757 tokens are new, and 29% of the original step was whole-cache work repeated in every call. We move that work to write time (a ring cache, RoPE and FP8 quantization applied once per write, and a delta rotation when positions reset), fuse the tensor-parallel all-reduce into the FP8 GEMM with in-switch reduction, and give the decoder that turns latents into video frames its own GPU. Without the fused kernel, our optimizations cut the diffusion step from 240.9 to 159.3 ms (1.51$\times$, 24.5 fps sustained) with no detected degradation in lip-sync scores; the fused kernel, rolled back once in production and since re-enabled, reaches 147.4 ms (1.63$\times$, 43% of FP8 peak). In seven weeks of production the worker averaged above 23.5 fps in 99.5% of sessions that streamed for at least a second, and our pipeline takes 0.49 s of a 2.1 s median conversational turn. On the same GPUs, the two open-source real-time video systems we benchmarked, at four denoising passes, miss real time or add over a second of latency (one by estimate), at a third to half of our resolution. ## Anthropic found a way to prove which AI model answered you -- but there's a costly catch DevFeed: [Anthropic found a way to prove which AI model answered you -- but there's a costly catch]() Original publisher: [Read original article]() Author: Amanda Caswell Published: 2026-11-04T22:50:18Z Content type: news Language: en Sources: [The New Stack]() Topics: [Zero-knowledge proof](), [Claude]() Tags: [ai](), [ai-models](), [anthropic](), [cryptographic](), [flashprove](), [github](), [gpu](), [h100](), [inference](), [lean](), [llama-4-maverick](), [model-weights](), [open-source](), [security]() ### AI overview FlashProve is an open-source research prototype that uses zkSNARKs to verify a response against a committed set of model weights without revealing the weights. Anthropic tested it on Meta's Llama 4 Maverick, not Claude: on one Nvidia H100 it processed about 110 prompt tokens per second while generating proofs. Each proof was about 18.78 MB and took roughly 21 seconds to verify on one CPU thread. Proof generation required 70-95 times more GPU time than standard vLLM inference, and proofs are generated in batches after requests, making routine or immediate verification costly. The protocol's properties were formally checked in Lean; FlashProve also includes a faster C++ verifier and a GPU prover, neither formally verified. The prototype is limited to 1,024 context tokens, integer inference, and greedy decoding, and is not a Claude API feature. ### Source excerpt Developers using AI APIs have little way of knowing whether the model answering their requests is actually the one they're The post Anthropic found a way to prove which AI model answered you -- but there's a costly catch appeared first on The New Stack. ## FP4Quant: Efficient 4-Bit Communication Compression for Model-Parallel LLM Training DevFeed: [FP4Quant: Efficient 4-Bit Communication Compression for Model-Parallel LLM Training]() Original publisher: [Read original article]() Author: Man Liu, Xingchen Liu, Hairui Zhao, Guangming Tan, Dingwen Tao Published: 2026-11-04T05:00:00Z Content type: article Language: en Sources: [arXiv: Artificial Intelligence]() Topics: [LLM optimization](), [quantization](), [distributed-training](), [Compression](), [GPU optimization](), [Large Language Model](), [LLM Techniques](), [Distributed Systems & Parallel Computing](), [qwen]() Tags: [communication](), [compression](), [cs-ai](), [distributed](), [fp4](), [gpu](), [h100](), [hybrid](), [llm](), [llm-training](), [packing](), [parallel](), [quality](), [quantization](), [qwen](), [scale](), [throughput](), [training]() ### AI overview FP4Quant targets communication overhead in model-parallel LLM training with a micro-grouped FP4 compression framework. It combines dual-scale quantization to preserve local tensor distributions, warp-cooperative packing for sub-byte data, and a GPU-aware two-level FWHT pipeline to reduce quantization error. On NVIDIA H100 GPUs with Qwen-7B, the authors report up to 1.56x end-to-end throughput versus BF16, with a trade-off between throughput and model quality. The excerpt does not quantify that quality trade-off. ### Source excerpt arXiv:2611.04323v1 Announce Type: new Abstract: The rapid scaling of large language models (LLMs) has made communication overhead a critical bottleneck in distributed model-parallel training. Although existing low-precision compression methods can effectively reduce communication costs, further reducing the communication footprint to ultra-low-bit formats such as FP4 introduces new challenges, including limited numerical range, sub-byte storage overhead, and increased computational cost from fine-grained quantization. In this paper, we present \textbf{FP4Quant}, a micro-grouped FP4 compression framework for efficient model-parallel LLM training. FP4Quant integrates a micro-grouped dual-scale FP4 quantization mechanism to preserve local tensor distribution characteristics, a warp-cooperative FP4 packing scheme to enable efficient sub-byte data representation, and a hybrid two-level FWHT pipeline to mitigate quantization errors with GPU-aware execution optimization. Experiments on NVIDIA H100 GPUs with Qwen-7B demonstrate up to 1.56$\times$ end-to-end throughput improvement over BF16, revealing a trade-off between throughput and model quality.