# NCCL

NVIDIA's library of topology-aware communication primitives for multi-GPU and multi-node applications.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## RoCE and NVIDIA Quantum InfiniBand, and their quantified tradeoffs

DevFeed: [RoCE and NVIDIA Quantum InfiniBand, and their quantified tradeoffs](<https://devfeed.tech/articles/roce-and-nvidia-quantum-infiniband-and-their-quantified-tradeoffs-146467.md>)

Original publisher: [Read original article](<https://lambda.com/blog/roce-and-nvidia-quantum-infiniband-and-their-quantified-tradeoffs>)

Author: Zach Mueller

Published: 2026-11-20T14:08:48Z

Content type: comparison

Language: en

Sources: [The Lambda deep learning blog](<https://devfeed.tech/sources/the-lambda-deep-learning-blog.md>)

Topics: [InfiniBand](<https://devfeed.tech/topics/infiniband.md>), [rdma](<https://devfeed.tech/topics/rdma.md>), [Distributed Systems & Parallel Computing](<https://devfeed.tech/topics/distributed-systems-parallel-computing.md>), [NCCL](<https://devfeed.tech/topics/nccl.md>), [Nvidia](<https://devfeed.tech/topics/nvidia.md>)

Tags: [1-click-cluster](<https://devfeed.tech/tags/1-click-cluster.md>), [cluster](<https://devfeed.tech/tags/cluster.md>), [infiniband](<https://devfeed.tech/tags/infiniband.md>), [llm-inference](<https://devfeed.tech/tags/llm-inference.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [quantum](<https://devfeed.tech/tags/quantum.md>), [roce](<https://devfeed.tech/tags/roce.md>), [tradeoffs](<https://devfeed.tech/tags/tradeoffs.md>)

### AI overview

RoCEv2 uses Ethernet with UDP/IP and can span subnets, while NVIDIA Quantum InfiniBand uses a dedicated fabric that requires InfiniBand switches and infrastructure. The text gives typical latency ranges of 0.5-2 µs for InfiniBand and 1-3 µs for optimized local RoCE, noting RoCE is more sensitive to configuration, congestion, and topology. In a two-node HGX B200 inference test using GLM-5.2-NVFP4 and SGLang, RoCE showed roughly 70% lower throughput than InfiniBand at about 250 concurrent users and 2.5x slower time to first token. The authors caution that real-world performance varies and recommend tempering expectations for On-Demand nodes; the comparison describes one setup.

### Source excerpt

The GPUs you rent are just part of what determines your cluster's speed and efficiency. This comes into play through interconnect, the network that connects multiple nodes (machines) on the backend.

## Benchmarking LLM Inference on Desktop-Scale NVIDIA Grace Blackwell Clusters

DevFeed: [Benchmarking LLM Inference on Desktop-Scale NVIDIA Grace Blackwell Clusters](<https://devfeed.tech/articles/benchmarking-llm-inference-on-desktop-scale-nvidia-grace-blackwell-clusters-136990.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.20289>)

Author: Kevin Bhimani, Christian Escobar, Daniel Mihovch, Rudolf Eigenmann, Sunita Chandrasekaran

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Distributed Computing](<https://devfeed.tech/sources/arxiv-distributed-computing.md>)

Topics: [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>), [model-serving](<https://devfeed.tech/topics/model-serving.md>), [Distributed Systems & Parallel Computing](<https://devfeed.tech/topics/distributed-systems-parallel-computing.md>), [vllm](<https://devfeed.tech/topics/vllm.md>), [DGX Spark](<https://devfeed.tech/topics/dgx-spark.md>), [NCCL](<https://devfeed.tech/topics/nccl.md>), [NVFP4](<https://devfeed.tech/topics/nvfp4.md>), [quantization](<https://devfeed.tech/topics/quantization.md>), [NVIDIA Blackwell](<https://devfeed.tech/topics/nvidia-blackwell.md>)

Tags: [17](<https://devfeed.tech/tags/17.md>), [benchmarking](<https://devfeed.tech/tags/benchmarking.md>), [bf16](<https://devfeed.tech/tags/bf16.md>), [blackwell](<https://devfeed.tech/tags/blackwell.md>), [clusters](<https://devfeed.tech/tags/clusters.md>), [collective](<https://devfeed.tech/tags/collective.md>), [communication](<https://devfeed.tech/tags/communication.md>), [concurrency](<https://devfeed.tech/tags/concurrency.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-dc](<https://devfeed.tech/tags/cs-dc.md>), [deployment](<https://devfeed.tech/tags/deployment.md>), [desktop](<https://devfeed.tech/tags/desktop.md>), [dgx-spark](<https://devfeed.tech/tags/dgx-spark.md>), [energy](<https://devfeed.tech/tags/energy.md>), [gb10](<https://devfeed.tech/tags/gb10.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [grace-blackwell](<https://devfeed.tech/tags/grace-blackwell.md>), [guidance](<https://devfeed.tech/tags/guidance.md>), [inference](<https://devfeed.tech/tags/inference.md>), [llm](<https://devfeed.tech/tags/llm.md>), [llm-inference](<https://devfeed.tech/tags/llm-inference.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>)

### AI overview

Benchmarks of 17 open-weight models on one or two RDMA-connected NVIDIA DGX Spark GB10 systems find that deployment strategy changes the benefit of adding a second system. With vLLM, tensor parallelism (TP=2) delivered a median 1.59x peak-throughput gain over one Spark, compared with 1.39x for pipeline parallelism (PP=2); TP=2 also reduced unloaded inter-token latency to 0.52-0.73x of one Spark, while PP=2 offered little reduction. For models that fit on one Spark, TP=2 generally suited low concurrency, whereas two replicas achieved higher throughput at high concurrency. One Spark used less GPU energy per output token at matched concurrency. The abstract also reports that NVFP4 reduced unloaded latency more than adding a second Spark at BF16, and suggests collective communication overhead helps explain TP=2's advantage.

### Source excerpt

arXiv:2611.20289v1 Announce Type: new Abstract: Two NVIDIA DGX Spark GB10 systems connect directly over 200 GbE to serve larger language models and increase inference capacity, but the benefits depend on how they are deployed. We benchmark 17 open-weight checkpoints spanning dense and mixture-of-experts architectures, 8B to 235B parameters and four weight precisions on one and two RDMA-connected systems. Using vLLM, we compare tensor parallelism (TP=2), pipeline parallelism (PP=2) and a two-replica baseline at concurrency 1 to 64, with three independent launches per configuration. TP=2 delivers a median 1.59x peak-throughput gain over one Spark, versus 1.39x for PP=2 and has higher throughput in 93 of 96 matched comparisons with separated 95% Welch intervals. It reduces unloaded inter-token latency to 0.52x to 0.73x of one Spark, whereas PP=2 provides little reduction. TP=2 also achieves the highest goodput under a strict 100 ms inter-token latency objective wherever both strategies are supported and the objective is met. For models fitting on one Spark, TP=2 generally favors low-concurrency serving, while replication offers higher throughput at high concurrency. One Spark uses less GPU energy per output token at matched concurrency. Across three precision ladders, NVFP4 reduces unloaded latency more than adding a second Spark at BF16. NCCL microbenchmarks and comparisons with an earlier TCP campaign suggest that collective communication overhead contributes to the advantage of TP=2. These results provide deployment guidance for desktop-scale inference.

## Zero-SM NCCL: A Systematic Guide to Designing and Deploying SM-Free GPU Communication

DevFeed: [Zero-SM NCCL: A Systematic Guide to Designing and Deploying SM-Free GPU Communication](<https://devfeed.tech/articles/zero-sm-nccl-a-systematic-guide-to-designing-and-deploying-sm-free-gpu-communication-134894.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.18075>)

Author: Zhenhao He, Siyuan Shen, Ke Wen, Prajwal Singhania, Shiqing Fan, Youngeun Kwon, Sangkug Lym, Vasudevan Rengasamy, Vishnuswaroop Ramesh, Yicheng Qiang, Craig Belusar, Jeff Hammond, Torsten Hoefler

Published: 2026-11-16T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Distributed Computing](<https://devfeed.tech/sources/arxiv-distributed-computing.md>)

Topics: [NCCL](<https://devfeed.tech/topics/nccl.md>), [rdma](<https://devfeed.tech/topics/rdma.md>), [Multi-GPU](<https://devfeed.tech/topics/multi-gpu.md>), [Distributed Systems & Parallel Computing](<https://devfeed.tech/topics/distributed-systems-parallel-computing.md>), [NVLink](<https://devfeed.tech/topics/nvlink.md>), [GPU](<https://devfeed.tech/topics/gpu.md>)

Tags: [accelerate](<https://devfeed.tech/tags/accelerate.md>), [apis](<https://devfeed.tech/tags/apis.md>), [bandwidth](<https://devfeed.tech/tags/bandwidth.md>), [building](<https://devfeed.tech/tags/building.md>), [communication](<https://devfeed.tech/tags/communication.md>), [copy](<https://devfeed.tech/tags/copy.md>), [cpu](<https://devfeed.tech/tags/cpu.md>), [cs-dc](<https://devfeed.tech/tags/cs-dc.md>), [cuda](<https://devfeed.tech/tags/cuda.md>), [data](<https://devfeed.tech/tags/data.md>), [data-movement](<https://devfeed.tech/tags/data-movement.md>), [deep](<https://devfeed.tech/tags/deep.md>), [designing](<https://devfeed.tech/tags/designing.md>), [domain](<https://devfeed.tech/tags/domain.md>), [domains](<https://devfeed.tech/tags/domains.md>), [free](<https://devfeed.tech/tags/free.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [guide](<https://devfeed.tech/tags/guide.md>), [nccl](<https://devfeed.tech/tags/nccl.md>), [sm](<https://devfeed.tech/tags/sm.md>), [zero](<https://devfeed.tech/tags/zero.md>)

### AI overview

Zero-SM NCCL avoids SM kernels by synchronizing with CUDA stream memory operations, using copy engines for transfers within a scale-up domain, and driving network RDMA through a stream-ordered CPU proxy. It supports AllGather and AlltoAll, plus one-sided ncclPutSignal and ncclWaitSignal APIs. On GB200 NVL72 and B200 NVL8, collectives achieved up to 1.36x NCCL's legacy AllGather bandwidth and reduced median overlapped GEMM slowdown from 33% to 4%, with higher per-operation latency. In Megatron-LM, FSDP improved by up to 3.5% and pipeline parallelism by up to 15% with one-sided puts. The gains depend on workload: the approach benefits large transfers in GPU-bound workloads and deep pipelines with large bubbles, but loses on small messages and host-bound workloads.

### Source excerpt

arXiv:2611.18075v1 Announce Type: new Abstract: GPU communication libraries such as NCCL use Streaming Multiprocessors (SMs) for data movement and synchronization, slowing overlapped computation at a cost that grows with interconnect bandwidth. We present Zero-SM NCCL, which communicates within and across scale-up domains without launching any SM kernel. It synchronizes through CUDA stream memory operations, moves data asynchronously with copy engines within a scale-up domain, and drives network RDMA from a stream-ordered CPU proxy. It supports data-movement collectives such as AllGather and AlltoAll, plus one-sided ncclPutSignal and ncclWaitSignal APIs. On GB200 NVL72 and B200 NVL8 systems, Zero-SM collectives reach up to 1.36x higher AllGather bandwidth than NCCL's legacy SM-based kernels and cut the median slowdown of overlapped GEMMs from 33% to 4%, at a higher latency per operation. In Megatron-LM training, they accelerate FSDP by up to 3.5%, and one-sided puts accelerate pipeline parallelism by up to 15%. Zero-SM communication is not a universal win, however. Through an end-to-end study across models and parallelism strategies, we show that it pays off for large transfers in GPU-bound workloads and for deep pipelines with large bubbles, but loses on small messages and host-bound workloads. We distill these findings into guidelines for enabling zero-SM communication in NCCL and building it into other libraries.

## HetCCL: Efficient LLM Training on Heterogeneous Vendor GPUs

DevFeed: [HetCCL: Efficient LLM Training on Heterogeneous Vendor GPUs](<https://devfeed.tech/articles/hetccl-efficient-llm-training-on-heterogeneous-vendor-gpus-134880.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2601.22585>)

Author: Heehoon Kim, Chris Jaehwan Lee, Taejeoung Kim, Jongwon Park, Jinpyo Kim, Pyongwon Suh, Ryan H. Choi, Sangwoo Lee, Jaejin Lee

Published: 2026-11-16T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Distributed Computing](<https://devfeed.tech/sources/arxiv-distributed-computing.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [distributed-training](<https://devfeed.tech/topics/distributed-training.md>), [NCCL](<https://devfeed.tech/topics/nccl.md>), [Multi-GPU](<https://devfeed.tech/topics/multi-gpu.md>), [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>)

Tags: [amd](<https://devfeed.tech/tags/amd.md>), [collective](<https://devfeed.tech/tags/collective.md>), [communication](<https://devfeed.tech/tags/communication.md>), [cs-dc](<https://devfeed.tech/tags/cs-dc.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [gpus](<https://devfeed.tech/tags/gpus.md>), [llm-training](<https://devfeed.tech/tags/llm-training.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [rdma](<https://devfeed.tech/tags/rdma.md>)

### AI overview

HetCCL enables collective communication across NVIDIA and AMD GPUs using RDMA, without driver modifications or changes to existing deep learning applications. It unifies vendor-specific backends and introduces two mechanisms that leverage NVIDIA NCCL and AMD RCCL. In the reported multi-vendor cluster evaluation, HetCCL matched NCCL and RCCL performance in homogeneous setups and scaled in heterogeneous environments. The supplied text does not describe the mechanisms or provide detailed evaluation results.

### Source excerpt

arXiv:2601.22585v2 Announce Type: replace Abstract: The rapid growth of large language models is driving organizations to expand their GPU clusters, often with GPUs from multiple vendors. However, current deep learning frameworks lack support for collective communication across heterogeneous GPUs, leading to inefficiency and higher costs. We present HetCCL, a collective communication library that unifies vendor-specific backends and enables RDMA-based communication across GPUs without requiring driver modifications. HetCCL introduces two novel mechanisms that enable cross-vendor communication while leveraging optimized vendor libraries, NVIDIA NCCL and AMD RCCL. Evaluations on a multi-vendor GPU cluster show that HetCCL matches NCCL and RCCL performance in homogeneous setups while uniquely scaling in heterogeneous environments, enabling practical, high-performance training with both NVIDIA and AMD GPUs without changes to existing deep learning applications.

## CoreWeave Unleashes the Power of the NVIDIA GB200 NVL72: A Glimpse into the Future of AI

DevFeed: [CoreWeave Unleashes the Power of the NVIDIA GB200 NVL72: A Glimpse into the Future of AI](<https://devfeed.tech/articles/coreweave-unleashes-the-power-of-the-nvidia-gb200-nvl72-a-glimpse-into-the-future-of-ai-133531.md>)

Original publisher: [Read original article](<https://wf.coreweave.com/blog/coreweave-unleashes-the-power-of-the-nvidia-gb200-nvl72-a-glimpse-into-the-future-of-ai>)

Author: CoreWeave

Published: 2026-11-13T20:57:46Z

Content type: article

Language: en

Sources: [Coreweave Blog Posts](<https://devfeed.tech/sources/coreweave-blog-posts.md>)

Topics: [GB200](<https://devfeed.tech/topics/gb200.md>), [Nvidia](<https://devfeed.tech/topics/nvidia.md>), [NVLink](<https://devfeed.tech/topics/nvlink.md>), [NCCL](<https://devfeed.tech/topics/nccl.md>), [Distributed Systems & Parallel Computing](<https://devfeed.tech/topics/distributed-systems-parallel-computing.md>), [observability](<https://devfeed.tech/topics/observability.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [blog](<https://devfeed.tech/tags/blog.md>), [cooling](<https://devfeed.tech/tags/cooling.md>), [coreweave](<https://devfeed.tech/tags/coreweave.md>), [efficiency](<https://devfeed.tech/tags/efficiency.md>), [energy](<https://devfeed.tech/tags/energy.md>), [energy-efficiency](<https://devfeed.tech/tags/energy-efficiency.md>), [future](<https://devfeed.tech/tags/future.md>), [gb200](<https://devfeed.tech/tags/gb200.md>), [infiniband](<https://devfeed.tech/tags/infiniband.md>), [innovation](<https://devfeed.tech/tags/innovation.md>), [kubernetes](<https://devfeed.tech/tags/kubernetes.md>), [networking](<https://devfeed.tech/tags/networking.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [nvidia-gb200](<https://devfeed.tech/tags/nvidia-gb200.md>), [nvl72](<https://devfeed.tech/tags/nvl72.md>), [observability](<https://devfeed.tech/tags/observability.md>), [performance](<https://devfeed.tech/tags/performance.md>), [power](<https://devfeed.tech/tags/power.md>), [quantum](<https://devfeed.tech/tags/quantum.md>), [support](<https://devfeed.tech/tags/support.md>)

### AI overview

CoreWeave demonstrated an NVIDIA GB200 NVL72 rack with distributed GPU benchmarks and a Megatron training run using CoreWeave Kubernetes Service (CKS) and Slurm on Kubernetes (SUNK). The company says CKS provides hypervisor-free Kubernetes tailored to AI workloads, while SUNK adds burst and batch job scheduling. CoreWeave Observability Services tracked GPU use, memory, power, and temperatures; dashboards also showed cooling output responding to workload and energy use scaling with performance. The rack uses Quantum-2 InfiniBand with 400 Gb/s per GPU and SHARP in-network collective communications. CoreWeave says it plans to deploy Quantum-X800 at 800 Gb/s per GPU. The text describes a live demonstration but provides no benchmark results or quantified efficiency comparisons.

### Source excerpt

We showcased NVIDIA GB200 NVL72 AI performance, energy efficiency, and cooling innovation, backed by Kubernetes support, observability tools, and Quantum-2 InfiniBand networking.

## When One GPU Isn't Enough: Serving GLM-5.3 Across Two NVIDIA DGX Stations with vLLM

DevFeed: [When One GPU Isn't Enough: Serving GLM-5.3 Across Two NVIDIA DGX Stations with vLLM](<https://devfeed.tech/articles/when-one-gpu-isn-t-enough-serving-glm-5-3-across-two-nvidia-dgx-stations-with-vllm-132447.md>)

Original publisher: [Read original article](<https://developer.nvidia.com/blog/when-one-gpu-isnt-enough-serving-glm-5-3-across-two-nvidia-dgx-stations-with-vllm/>)

Author: Fatima Fall

Published: 2026-11-13T17:00:00Z

Content type: tutorial

Language: en

Sources: [NVIDIA Developer](<https://devfeed.tech/sources/nvidia-developer.md>), [NVIDIA Technical Blog](<https://devfeed.tech/sources/nvidia-technical-blog.md>)

Topics: [vllm](<https://devfeed.tech/topics/vllm.md>), [DGX Station](<https://devfeed.tech/topics/dgx-station.md>), [gpudirect rdma](<https://devfeed.tech/topics/gpudirect-rdma.md>), [NCCL](<https://devfeed.tech/topics/nccl.md>), [Benchmark](<https://devfeed.tech/topics/benchmark.md>), [Multi-GPU](<https://devfeed.tech/topics/multi-gpu.md>), [quantization](<https://devfeed.tech/topics/quantization.md>), [NVFP4](<https://devfeed.tech/topics/nvfp4.md>)

Tags: [agentic-ai-generative-ai](<https://devfeed.tech/tags/agentic-ai-generative-ai.md>), [agents](<https://devfeed.tech/tags/agents.md>), [api](<https://devfeed.tech/tags/api.md>), [benchmarking](<https://devfeed.tech/tags/benchmarking.md>), [dgx](<https://devfeed.tech/tags/dgx.md>), [dgx-station](<https://devfeed.tech/tags/dgx-station.md>), [edge-computing](<https://devfeed.tech/tags/edge-computing.md>), [glm-5-3](<https://devfeed.tech/tags/glm-5-3.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [gpu-memory](<https://devfeed.tech/tags/gpu-memory.md>), [inference-performance](<https://devfeed.tech/tags/inference-performance.md>), [llm-benchmarking](<https://devfeed.tech/tags/llm-benchmarking.md>), [llms](<https://devfeed.tech/tags/llms.md>), [local-ai](<https://devfeed.tech/tags/local-ai.md>), [networking-communications](<https://devfeed.tech/tags/networking-communications.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [serving](<https://devfeed.tech/tags/serving.md>), [vllm](<https://devfeed.tech/tags/vllm.md>)

### AI overview

The tutorial runs the 753-billion-parameter GLM-5.3 NVFP4 model across two GB300 DGX Stations because its roughly 464 GB checkpoint exceeds one GPU's 252 GB of memory. vLLM tensor parallelism splits weights and computation across both GPUs; ConnectX-8 RDMA carries rank-to-rank traffic, and Station A exposes a single OpenAI-compatible API. The workflow covers validating the link, downloading and syncing identical checkpoint copies, launching one vLLM rank per station, checking RDMA and API health, and measuring load with AIPerf on 80 SPEED-Bench coding conversations. Prerequisites include substantial disk space on each station, SSH access, Hugging Face access to the gated dataset, and acceptance of the model and dataset terms. Reported throughput peaks at 607.8 tokens/s at concurrency 16, then drops to 385.0 tokens/s at concurrency 32, where mean latency is 453.5 seconds; mean per-user throughput declines as concurrency rises. The excerpt notes that longer prompts or more concurrent users need more KV cache, and suggests reducing context length or request concurrency if memory runs out.

### Source excerpt

Running a frontier-scale model locally means keeping all its weights in GPU memory; the largest models no longer fit on one machine. Two NVIDIA DGX Stations...

## Network Contention State Aware Dynamic Optimizer for Accelerator Collective Communication Libraries

DevFeed: [Network Contention State Aware Dynamic Optimizer for Accelerator Collective Communication Libraries](<https://devfeed.tech/articles/network-contention-state-aware-dynamic-optimizer-for-accelerator-collective-communication-libraries-125292.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.10984>)

Author: Rajat Bisht (Tennessee Technological University), Tevfik Kosar (University at Buffalo), Anthony Skjellum (Tennessee Technological University), Gene Cooperman (Northeastern University), MD S Q Zulkar N

Published: 2026-11-10T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Distributed Computing](<https://devfeed.tech/sources/arxiv-distributed-computing.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Multi Agent Systems](<https://devfeed.tech/topics/multi-agent-systems.md>), [NCCL](<https://devfeed.tech/topics/nccl.md>), [distributed-training](<https://devfeed.tech/topics/distributed-training.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [agent-architecture](<https://devfeed.tech/tags/agent-architecture.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-ml](<https://devfeed.tech/tags/ai-ml.md>), [ai-workloads](<https://devfeed.tech/tags/ai-workloads.md>), [bare-metal](<https://devfeed.tech/tags/bare-metal.md>), [cloud](<https://devfeed.tech/tags/cloud.md>), [cluster](<https://devfeed.tech/tags/cluster.md>), [collective](<https://devfeed.tech/tags/collective.md>), [communication](<https://devfeed.tech/tags/communication.md>), [configuration](<https://devfeed.tech/tags/configuration.md>), [connectivity](<https://devfeed.tech/tags/connectivity.md>), [cs-dc](<https://devfeed.tech/tags/cs-dc.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-ma](<https://devfeed.tech/tags/cs-ma.md>), [delta](<https://devfeed.tech/tags/delta.md>), [fabric](<https://devfeed.tech/tags/fabric.md>), [gpus](<https://devfeed.tech/tags/gpus.md>), [latency](<https://devfeed.tech/tags/latency.md>), [learning](<https://devfeed.tech/tags/learning.md>), [libraries](<https://devfeed.tech/tags/libraries.md>), [long-short-term-memory](<https://devfeed.tech/tags/long-short-term-memory.md>), [network](<https://devfeed.tech/tags/network.md>)

### AI overview

Static collective communication settings can degrade under background network traffic. This NCCL runtime tuner uses an LSTM-based detector with a sequential probability ratio test to distinguish sustained contention from transient spikes, then a Dueling Double DQN selector with Conservative Q-Learning to adjust configuration parameters. On Chameleon Cloud's isolated bare-metal cluster, it reduced AllReduce latency by 36% versus NCCL and 32% versus fully converged AutoCCL. Across six AI workloads, it reported up to 65% speedups on Delta with NVIDIA A100 GPUs and Slingshot-11. The supplied abstract does not specify the workload-level results or broader evaluation limits.

### Source excerpt

arXiv:2611.10984v1 Announce Type: cross Abstract: Collective Communication Libraries (CCL) are critical for large-scale distributed AI/ML training, yet their configurations are determined by static heuristics that do not account for real-time network conditions. Our empirical study shows that these heuristics perform near-optimally in contention-free environments, but degrade substantially under background traffic. To address this, we present a cooperative multi-agent run-time optimizer for accelerator collective communication. Our approach employs a two-agent architecture: (i) detector- a Long Short Term Memory(LSTM) recurring neural network fed classifier head with a sequential probability ratio test that distinguishes sustained contention from transient spikes; and (ii) selector- a configuration selector built on a Dueling Double DQN (D3QN) with Conservative Q-Learning that adapts configuration parameters in response. We implemented this dynamic tuner over NCCL, a widely used CCL developed by NVIDIA. When evaluated on Chameleon Cloud's isolated bare-metal cluster, this dynamic tuning approach reduces AllReduce latency by 36% over NCCL and 32% over a fully converged AutoCCL. We further evaluated it across six real AI workloads spanning convolutional networks, transformers, and recommendation architectures. Our multi-agent tuner approach delivers up to 65% speedups on Delta cluster with NVIDIA A100 GPUs and Slingshot-11 fabric internode connectivity.

## PyTorch 2.15 tackles the 'one bad GPU' problem

DevFeed: [PyTorch 2.15 tackles the 'one bad GPU' problem](<https://devfeed.tech/articles/pytorch-2-15-tackles-the-one-bad-gpu-problem-122495.md>)

Original publisher: [Read original article](<https://www.devclass.com/devops/2026/11/07/pytorch-215-tackles-the-one-bad-gpu-problem/5308234>)

Author: Joab Jackson

Published: 2026-11-07T18:38:00Z

Content type: news

Language: en

Sources: [DevClass.com](<https://devfeed.tech/sources/devclass-com.md>)

Topics: [PyTorch](<https://devfeed.tech/topics/pytorch.md>), [NCCL](<https://devfeed.tech/topics/nccl.md>), [Multi-GPU](<https://devfeed.tech/topics/multi-gpu.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [api](<https://devfeed.tech/tags/api.md>), [apple](<https://devfeed.tech/tags/apple.md>), [backend](<https://devfeed.tech/tags/backend.md>), [blackwell](<https://devfeed.tech/tags/blackwell.md>), [bottlenecks](<https://devfeed.tech/tags/bottlenecks.md>), [cluster](<https://devfeed.tech/tags/cluster.md>), [clusters](<https://devfeed.tech/tags/clusters.md>), [communication](<https://devfeed.tech/tags/communication.md>), [communications](<https://devfeed.tech/tags/communications.md>), [computing](<https://devfeed.tech/tags/computing.md>), [data-movement](<https://devfeed.tech/tags/data-movement.md>), [deadlocks](<https://devfeed.tech/tags/deadlocks.md>), [devops](<https://devfeed.tech/tags/devops.md>), [distributed](<https://devfeed.tech/tags/distributed.md>), [distributed-computing](<https://devfeed.tech/tags/distributed-computing.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [job](<https://devfeed.tech/tags/job.md>), [ml](<https://devfeed.tech/tags/ml.md>), [nccl](<https://devfeed.tech/tags/nccl.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [pytorch](<https://devfeed.tech/tags/pytorch.md>), [training](<https://devfeed.tech/tags/training.md>)

### AI overview

PyTorch 2.15 rewrites its NCCL backend to improve diagnosis and recovery in multi-GPU jobs. The former ProcessGroupNCCL could create many communicators per process group, contributing to intermittent deadlocks and unpredictable performance; the new backend limits each group to one communicator at initialization, so programs needing parallelism must dispatch more groups. It also supports easier reconfiguration when a GPU is slow or fails: a third-party orchestrator can exclude it and assign a replacement worker. A new flight recorder stores operation snapshots for inspection after failures. The legacy backend remains available as a rollback in 2.15 and is slated for removal in 2.16 if the replacement works as expected. Other reported additions include an asynchronous one-way Python transport API and parallel-processing improvements for Apple hardware and Nvidia Blackwell GPUs.

### Source excerpt

ML admins will be happy to learn that one bad GPU may no longer mean starting an entire training job ...

## Characterizing CUDA Locality Domains on Blackwell GPUs

DevFeed: [Characterizing CUDA Locality Domains on Blackwell GPUs](<https://devfeed.tech/articles/characterizing-cuda-locality-domains-on-blackwell-gpus-121269.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.07535>)

Author: Rocky Wang, Joyce Zhou, Keith Jiang, Jason Zhao

Published: 2026-11-06T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Distributed Computing](<https://devfeed.tech/sources/arxiv-distributed-computing.md>)

Topics: [CUDA](<https://devfeed.tech/topics/cuda.md>), [GPU optimization](<https://devfeed.tech/topics/gpu-optimization.md>), [Multi-GPU](<https://devfeed.tech/topics/multi-gpu.md>), [NVIDIA Blackwell](<https://devfeed.tech/topics/nvidia-blackwell.md>), [NCCL](<https://devfeed.tech/topics/nccl.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [also](<https://devfeed.tech/tags/also.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [b300](<https://devfeed.tech/tags/b300.md>), [blackwell](<https://devfeed.tech/tags/blackwell.md>), [can](<https://devfeed.tech/tags/can.md>), [changes](<https://devfeed.tech/tags/changes.md>), [code](<https://devfeed.tech/tags/code.md>), [communication](<https://devfeed.tech/tags/communication.md>), [costs](<https://devfeed.tech/tags/costs.md>), [critical](<https://devfeed.tech/tags/critical.md>), [cross](<https://devfeed.tech/tags/cross.md>), [cs-dc](<https://devfeed.tech/tags/cs-dc.md>), [cuda](<https://devfeed.tech/tags/cuda.md>), [data](<https://devfeed.tech/tags/data.md>), [deepgemm](<https://devfeed.tech/tags/deepgemm.md>), [domain](<https://devfeed.tech/tags/domain.md>), [domains](<https://devfeed.tech/tags/domains.md>), [driver](<https://devfeed.tech/tags/driver.md>), [gpus](<https://devfeed.tech/tags/gpus.md>), [locality](<https://devfeed.tech/tags/locality.md>)

### AI overview

On one node with eight NVIDIA B300 GPUs, the study finds that CUDA 13.4 Locality Domains can reduce latency, but benefits depend on data access patterns. Die-aware adaptations reportedly improve NCCL 2.32.3 AllGather by up to 43.5%, nvbandwidth DtoD by 111.3%, and DeepGEMM EP by 37%. The abstract also reports allocation overhead for sizes above 32 MB, linked to barriers and TLB invalidation. The authors propose a GPU driver kernel module fix, submitted to NVIDIA, that they say could reduce this overhead 7-10x without user-code changes.

### Source excerpt

arXiv:2611.07535v1 Announce Type: new Abstract: Low latency is critical to user experience. As NVIDIA Blackwell and Rubin GPUs advance multi-die scaling, data placement and cross-die communication become increasingly important. However, the latency implications and underlying costs of cross-die communication and die-aware execution remain largely unexplored. In this report, we present a comprehensive evaluation of the Locality Domain (LD) feature introduced in CUDA 13.4. Conducted on a single node equipped with eight NVIDIA B300 GPUs, our study examines three aspects: memory management and access, computation, and communication, to identify the sources of latency and, more importantly, explore how LD can mitigate them. Our measurements show that the benefits of LD depend on data access patterns. Through die-aware adaptations and integration with widely used libraries and tools, we achieve speedups of up to 43.5% for NCCL 2.32.3 (AllGather), 111.3% for nvbandwidth (DtoD), and 37% for DeepGEMM (EP). We also identify non-negligible LD-related GPU memory allocation overhead when allocation sizes exceed 32 MB due to barrier and TLB invalidation, we propose a fix to the GPU driver kernel module (submitted to NVIDIA) and show that overhead could be reduced 7x ~ 10x without requiring any user-code changes.

## Configure, Certify, and Operate Kubernetes GPU Clusters with NVIDIA DSX OS

DevFeed: [Configure, Certify, and Operate Kubernetes GPU Clusters with NVIDIA DSX OS](<https://devfeed.tech/articles/configure-certify-and-operate-kubernetes-gpu-clusters-with-nvidia-dsx-os-119476.md>)

Original publisher: [Read original article](<https://developer.nvidia.com/blog/configure-certify-and-operate-kubernetes-gpu-clusters-with-nvidia-dsx-os/>)

Author: Sowmyan Soman

Published: 2026-11-05T17:00:00Z

Content type: tutorial

Language: en

Sources: [NVIDIA Developer](<https://devfeed.tech/sources/nvidia-developer.md>), [NVIDIA Technical Blog](<https://devfeed.tech/sources/nvidia-technical-blog.md>)

Topics: [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [gpu monitoring](<https://devfeed.tech/topics/gpu-monitoring.md>), [NVIDIA DSX](<https://devfeed.tech/topics/nvidia-dsx.md>), [NCCL](<https://devfeed.tech/topics/nccl.md>), [CI/CD](<https://devfeed.tech/topics/cicd.md>), [Open Source](<https://devfeed.tech/topics/open-source.md>)

Tags: [agentic-ai-generative-ai](<https://devfeed.tech/tags/agentic-ai-generative-ai.md>), [basic](<https://devfeed.tech/tags/basic.md>), [can](<https://devfeed.tech/tags/can.md>), [cloud-services](<https://devfeed.tech/tags/cloud-services.md>), [clusters](<https://devfeed.tech/tags/clusters.md>), [configure](<https://devfeed.tech/tags/configure.md>), [data-center-cloud](<https://devfeed.tech/tags/data-center-cloud.md>), [developer-tools-techniques](<https://devfeed.tech/tags/developer-tools-techniques.md>), [distributed](<https://devfeed.tech/tags/distributed.md>), [distributed-training](<https://devfeed.tech/tags/distributed-training.md>), [dsx](<https://devfeed.tech/tags/dsx.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [gpu-clusters](<https://devfeed.tech/tags/gpu-clusters.md>), [health](<https://devfeed.tech/tags/health.md>), [health-checks](<https://devfeed.tech/tags/health-checks.md>), [job](<https://devfeed.tech/tags/job.md>), [kubernetes](<https://devfeed.tech/tags/kubernetes.md>), [leaving](<https://devfeed.tech/tags/leaving.md>), [node](<https://devfeed.tech/tags/node.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [nvidia-dsx](<https://devfeed.tech/tags/nvidia-dsx.md>), [one](<https://devfeed.tech/tags/one.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [os](<https://devfeed.tech/tags/os.md>), [with](<https://devfeed.tech/tags/with.md>)

### AI overview

NVIDIA DSX OS combines three open source tools to keep Kubernetes GPU clusters configured, workload-certified, and healthy through changes and repairs. AICR builds version-locked configuration recipes, renders them for existing deployers, and can detect drift; it generates artifacts but does not apply them. NVCRE runs GPU, network, storage, and AI workload tests, reports measurements and suspected faulty nodes, and applies performance thresholds only when configured. NVSentinel continuously monitors existing health signals without using GPU time. It starts in reporting-only mode; when configured, it can quarantine or drain unhealthy nodes and coordinate NVCRE revalidation before they return to service. The tools divide responsibility: NVCRE records failures and attribution, while NVSentinel handles quarantine and repair. Lifecycle automation can validate new and remediated nodes, but driver, operator, or fabric changes need a separate pipeline-triggered validation process. The excerpt assumes the tools, cluster dependencies, and NVCRE test provider are installed and configured.

### Source excerpt

One underperforming GPU can slow an entire distributed training job. The node may pass basic health checks and show normal utilization, leaving operators to...

## flash-comm: Self-Synchronizing Collective Communication for LLM Decoding

DevFeed: [flash-comm: Self-Synchronizing Collective Communication for LLM Decoding](<https://devfeed.tech/articles/flash-comm-self-synchronizing-collective-communication-for-llm-decoding-119117.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.06634>)

Author: Carrie Wang, Jack Li, Jason Zhao

Published: 2026-11-05T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Distributed Computing](<https://devfeed.tech/sources/arxiv-distributed-computing.md>)

Topics: [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>), [NCCL](<https://devfeed.tech/topics/nccl.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>)

Tags: [asynchronous](<https://devfeed.tech/tags/asynchronous.md>), [b300](<https://devfeed.tech/tags/b300.md>), [collective](<https://devfeed.tech/tags/collective.md>), [communication](<https://devfeed.tech/tags/communication.md>), [consistency](<https://devfeed.tech/tags/consistency.md>), [cost](<https://devfeed.tech/tags/cost.md>), [cs-dc](<https://devfeed.tech/tags/cs-dc.md>), [data-movement](<https://devfeed.tech/tags/data-movement.md>), [decoding](<https://devfeed.tech/tags/decoding.md>), [encoding](<https://devfeed.tech/tags/encoding.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [gpus](<https://devfeed.tech/tags/gpus.md>), [latency](<https://devfeed.tech/tags/latency.md>), [library](<https://devfeed.tech/tags/library.md>), [llm](<https://devfeed.tech/tags/llm.md>), [median](<https://devfeed.tech/tags/median.md>), [message](<https://devfeed.tech/tags/message.md>), [protocol](<https://devfeed.tech/tags/protocol.md>), [reduction](<https://devfeed.tech/tags/reduction.md>), [selector](<https://devfeed.tech/tags/selector.md>), [synchronization](<https://devfeed.tech/tags/synchronization.md>)

### AI overview

flash-comm fuses readiness synchronization with data movement: each data block carries readiness evidence so receivers can validate needed blocks without separate inter-GPU barriers. A cost-guided selector adapts readiness encoding to message size and GPU count, balancing synchronization overhead against extra transfer cost. It also separates transport order from reduction order to preserve result consistency with asynchronous arrivals. On eight NVIDIA B300 GPUs, all-reduce latency fell by a median 28.8% and up to 43.7% versus custom AR v2; end-to-end decoding time per output token fell 1.7-6.8%. Since all-reduce was 11.4% of baseline decoding time, optimizing it alone limits possible overall latency gains. Standalone all-gather latency fell by a median 23.8% and up to 58.9%.

### Source excerpt

arXiv:2611.06634v1 Announce Type: new Abstract: We present flash-comm, a communication library that achieves efficient transport for LLM decoding by fusing readiness synchronization with data movement. flash-comm introduces self-synchronizing transfers in which each data block carries evidence of its readiness, allowing receivers to directly validate the blocks they need without separate inter-GPU barriers. Based on this principle, a cost-guided protocol selector adapts the readiness encoding to message size and the number of participating GPUs, balancing synchronization overhead against additional data-transfer cost. flash-comm further decouples transport order from reduction order to preserve result consistency across GPUs under asynchronous data arrivals. On a server with eight NVIDIA B300 GPUs, flash-comm reduces all-reduce latency by a median of 28.8% and up to 43.7% compared with custom AR v2 (the state-of-the-art all-reduce implementation). In end-to-end decoding, flash-comm reduces time per output token (TPOT) by 1.7-6.8%. Note that all-reduce accounts for 11.4% of the baseline decoding time, which bounds the latency reduction possible from optimizing this operation alone. The same mechanism also supports standalone all-gather, reducing latency by a median of 23.8% and up to 58.9% across eight B300 GPUs.

## Dual-Node NVIDIA DGX Spark over Tailscale: A Remote-Access Testbed for Distributed LLM Training and Cyber-Threat-Intelligence Fine-Tuning

DevFeed: [Dual-Node NVIDIA DGX Spark over Tailscale: A Remote-Access Testbed for Distributed LLM Training and Cyber-Threat-Intelligence Fine-Tuning](<https://devfeed.tech/articles/dual-node-nvidia-dgx-spark-over-tailscale-a-remote-access-testbed-for-distributed-llm-training-and-cyber-threat-intelligence-fine-tuning-116734.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2608.07226>)

Author: Vasanth Iyer

Published: 2026-11-04T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [distributed-training](<https://devfeed.tech/topics/distributed-training.md>), [DGX Spark](<https://devfeed.tech/topics/dgx-spark.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [tailscale](<https://devfeed.tech/topics/tailscale.md>), [NCCL](<https://devfeed.tech/topics/nccl.md>), [Threat Hunting & Intel](<https://devfeed.tech/topics/threat-hunting-intel.md>)

Tags: [cs-ar](<https://devfeed.tech/tags/cs-ar.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cyber](<https://devfeed.tech/tags/cyber.md>), [cyber-threat-intelligence](<https://devfeed.tech/tags/cyber-threat-intelligence.md>), [dgx](<https://devfeed.tech/tags/dgx.md>), [dgx-spark](<https://devfeed.tech/tags/dgx-spark.md>), [distributed](<https://devfeed.tech/tags/distributed.md>), [fine](<https://devfeed.tech/tags/fine.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [intelligence](<https://devfeed.tech/tags/intelligence.md>), [llm](<https://devfeed.tech/tags/llm.md>), [llm-training](<https://devfeed.tech/tags/llm-training.md>), [node](<https://devfeed.tech/tags/node.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [nvidia-dgx](<https://devfeed.tech/tags/nvidia-dgx.md>), [over](<https://devfeed.tech/tags/over.md>), [remote](<https://devfeed.tech/tags/remote.md>), [remote-access](<https://devfeed.tech/tags/remote-access.md>), [spark](<https://devfeed.tech/tags/spark.md>), [tailscale](<https://devfeed.tech/tags/tailscale.md>), [testbed](<https://devfeed.tech/tags/testbed.md>), [training](<https://devfeed.tech/tags/training.md>)

### AI overview

Two NVIDIA DGX Spark systems completed distributed NanoChat pretraining over a dedicated 200 Gb/s fiber link, with remote administration through Tailscale. Using PyTorch torchrun, DDP, and NCCL, the 896.5-million-parameter run processed about 5.22 billion tokens in 4,980 steps over roughly four days, averaging 63.4 seconds per step. The report documents setup and troubleshooting, including an initial-evaluation bug that caused NCCL timeouts. In fine-tuning experiments, adding data from 77 CISA advisories improved CTI-specific scores but weakened general-knowledge performance, with a small overall score increase. A second stage using a MITRE ATT&CK technique dictionary achieved 98.5% accuracy on dictionary lookups but named none of 50 held-out technique IDs correctly; related held-out threat knowledge remained near zero, and general ability declined. The authors frame the deployment as evidence of feasibility, not scaling efficiency.

### Source excerpt

arXiv:2608.07226v2 Announce Type: replace-cross Abstract: Practical evidence for multi-node training on desktop-class accelerators remains limited. This report presents a proof-of-concept deployment of distributed NanoChat pretraining across two NVIDIA DGX Spark systems, each with a GB10 Grace Blackwell system-on-chip and 128 GB of unified memory, administered remotely over a Tailscale mesh VPN and connected for training by a dedicated 200 Gb/s QSFP56 direct fiber link. PyTorch torchrun, DDP, and NCCL were configured with one process per node, a depth-20 NanoChat model, a device batch of 32 sequences per node, a 2,048-token context, and eight gradient-accumulation steps, giving 1,048,576 tokens per optimizer step for the 896.5-million-parameter model. The run took 4,980 steps over about four days, averaging 63.4 s of measured training time per step (about 16,540 tokens/s) and processing about 5.22 billion tokens. We document link configuration, container setup, interface binding, a step-zero evaluation bug that triggered NCCL timeouts, checkpointing, and troubleshooting lessons. We also built a cybersecurity fine-tuning dataset from 77 CISA advisories (338 training, 37 validation conversations) and compared a baseline SFT checkpoint against a CTI-augmented checkpoint on 17 held-out prompts with an Ollama-hosted LLM judge: CTI-specific categories improved while general-knowledge categories regressed, for a small overall change from 2.06 to 2.29 on a 0-10 scale. Version 2 adds a stage-2 fine-tune with a MITRE ATT&CK technique dictionary under a deterministic evaluation: in-dictionary ID-to-name lookup reaches 98.5%, yet none of 50 held-out IDs is named correctly, knowledge of held-out threat groups, software, campaigns, and vulnerabilities stays near zero, and general ability drops measurably. The study establishes feasibility rather than a scaling-efficiency claim.

## Diagnose Distributed AI Workload Failures with NVIDIA Attribution

DevFeed: [Diagnose Distributed AI Workload Failures with NVIDIA Attribution](<https://devfeed.tech/articles/diagnose-distributed-ai-workload-failures-with-nvidia-attribution-113259.md>)

Original publisher: [Read original article](<https://developer.nvidia.com/blog/diagnose-distributed-ai-workload-failures-with-nvidia-attribution/>)

Author: Elizabeth Goodman

Published: 2026-11-02T23:39:34Z

Content type: tutorial

Language: en

Sources: [NVIDIA Developer](<https://devfeed.tech/sources/nvidia-developer.md>), [NVIDIA Technical Blog](<https://devfeed.tech/sources/nvidia-technical-blog.md>)

Topics: [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [distributed-training](<https://devfeed.tech/topics/distributed-training.md>), [NVLink](<https://devfeed.tech/topics/nvlink.md>), [Monitoring](<https://devfeed.tech/topics/monitoring.md>), [NCCL](<https://devfeed.tech/topics/nccl.md>)

Tags: [accelerated-computing-libraries](<https://devfeed.tech/tags/accelerated-computing-libraries.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-workloads](<https://devfeed.tech/tags/ai-workloads.md>), [data-center-cloud](<https://devfeed.tech/tags/data-center-cloud.md>), [developer-tools-techniques](<https://devfeed.tech/tags/developer-tools-techniques.md>), [diagnose](<https://devfeed.tech/tags/diagnose.md>), [distributed](<https://devfeed.tech/tags/distributed.md>), [error](<https://devfeed.tech/tags/error.md>), [errors](<https://devfeed.tech/tags/errors.md>), [fabric](<https://devfeed.tech/tags/fabric.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [mlops](<https://devfeed.tech/tags/mlops.md>), [nccl](<https://devfeed.tech/tags/nccl.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [nvl72](<https://devfeed.tech/tags/nvl72.md>), [scheduler](<https://devfeed.tech/tags/scheduler.md>)

### AI overview

NVIDIA Attribution diagnoses distributed AI workload failures by correlating workload-scoped logs, metrics, telemetry, and scheduler state with deterministic rules. The Blueprint scopes a workload, collects observations, and calls the Attribution Service, which extracts and correlates events; the Blueprint returns a structured verdict with outcome, attributed domain, affected nodes, symptoms, and collector coverage. In the example, XID 149 on node-02 is identified as the root cause of a signal-integrity failure, while downstream NCCL errors are retained as evidence. When evidence supports multiple causes, the result can mark the sub-domain ambiguous; diagnosis quality depends on telemetry completeness. The self-hosted HTTP Service runs the core engine, while the customizable Python Blueprint integrates schedulers and data sources. Its quickstart uses Docker and uv with sample evidence. Rules ship with the Service and are not uploaded dynamically through its API.

### Source excerpt

In distributed AI workloads, a single error can produce a cascade of symptoms. A GPU or fabric event may surface as NCCL errors on other ranks, scheduler state...

## Making Kubernetes AI-Ready for NVIDIA Vera Rubin on Day One

DevFeed: [Making Kubernetes AI-Ready for NVIDIA Vera Rubin on Day One](<https://devfeed.tech/articles/making-kubernetes-ai-ready-for-nvidia-vera-rubin-on-day-one-112593.md>)

Original publisher: [Read original article](<https://developer.nvidia.com/blog/making-kubernetes-ai-ready-for-nvidia-vera-rubin-on-day-one/>)

Author: Shara Tibken

Published: 2026-11-02T18:00:00Z

Content type: article

Language: en

Sources: [NVIDIA Developer](<https://devfeed.tech/sources/nvidia-developer.md>), [NVIDIA Technical Blog](<https://devfeed.tech/sources/nvidia-technical-blog.md>)

Topics: [NVIDIA Vera Rubin](<https://devfeed.tech/topics/nvidia-vera-rubin.md>), [Vera Rubin NVL72](<https://devfeed.tech/topics/vera-rubin-nvl72.md>), [NVIDIA DSX](<https://devfeed.tech/topics/nvidia-dsx.md>), [NVLink](<https://devfeed.tech/topics/nvlink.md>), [NCCL](<https://devfeed.tech/topics/nccl.md>), [DOCA](<https://devfeed.tech/topics/doca.md>), [GPUDirect](<https://devfeed.tech/topics/gpudirect.md>)

Tags: [2](<https://devfeed.tech/tags/2.md>), [3](<https://devfeed.tech/tags/3.md>), [about](<https://devfeed.tech/tags/about.md>), [agentic-ai-generative-ai](<https://devfeed.tech/tags/agentic-ai-generative-ai.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-factory](<https://devfeed.tech/tags/ai-factory.md>), [bluefield-dpu](<https://devfeed.tech/tags/bluefield-dpu.md>), [cloud-services](<https://devfeed.tech/tags/cloud-services.md>), [complex](<https://devfeed.tech/tags/complex.md>), [connectx](<https://devfeed.tech/tags/connectx.md>), [data-center-cloud](<https://devfeed.tech/tags/data-center-cloud.md>), [drivers](<https://devfeed.tech/tags/drivers.md>), [dsx](<https://devfeed.tech/tags/dsx.md>), [dsx-ai-factory-platform](<https://devfeed.tech/tags/dsx-ai-factory-platform.md>), [error](<https://devfeed.tech/tags/error.md>), [gb200](<https://devfeed.tech/tags/gb200.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [hardware](<https://devfeed.tech/tags/hardware.md>), [infrastructure](<https://devfeed.tech/tags/infrastructure.md>), [kernels](<https://devfeed.tech/tags/kernels.md>), [kubernetes](<https://devfeed.tech/tags/kubernetes.md>), [machine-learning-artificial-intelligence](<https://devfeed.tech/tags/machine-learning-artificial-intelligence.md>), [networking](<https://devfeed.tech/tags/networking.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [nvidia-vera-rubin](<https://devfeed.tech/tags/nvidia-vera-rubin.md>), [nvlink](<https://devfeed.tech/tags/nvlink.md>), [one](<https://devfeed.tech/tags/one.md>), [optimizing](<https://devfeed.tech/tags/optimizing.md>), [rubin](<https://devfeed.tech/tags/rubin.md>), [rubin-gpu](<https://devfeed.tech/tags/rubin-gpu.md>), [runtimes](<https://devfeed.tech/tags/runtimes.md>), [vera](<https://devfeed.tech/tags/vera.md>), [vera-rubin](<https://devfeed.tech/tags/vera-rubin.md>), [vera-rubin-nvl72](<https://devfeed.tech/tags/vera-rubin-nvl72.md>)

### AI overview

AICR captures Kubernetes settings for Vera Rubin NVL72 in reusable Preview recipes, then bundles, snapshots and validates the cluster. On early hardware, training and inference variants passed deployment and conformance checks; training also passed a cross-node NCCL all-reduce performance gate over NVLink through DRA-managed IMEX ComputeDomains. The validation used GPU Operator 26.7.0, DRA driver 0.5.0 and host-installed R615.23, with Ubuntu 26.04, a 64k-page kernel and Kubernetes 1.34.1 or later. Recipes use a host-managed GPU driver, DRA for ComputeDomains and whole-GPU allocation; MIG and ConnectX-9 RDMA data-path performance were not validated. Inference serving performance remains deferred pending a published runtime built with CUDA 13.4 or later and verified on hardware. Recipes are Preview, and published validation records apply to the recipe version captured, not necessarily its current content.

### Source excerpt

Configuring and optimizing Kubernetes for new GPU hardware such as NVIDIA Vera Rubin is complex and error-prone. Drivers, kernels, runtimes, networking,...

## The Endgame of FSDP Communication: Layout-Aware Collectives on Copy Engines, RDMA, and NVSwitch

DevFeed: [The Endgame of FSDP Communication: Layout-Aware Collectives on Copy Engines, RDMA, and NVSwitch](<https://devfeed.tech/articles/the-endgame-of-fsdp-communication-layout-aware-collectives-on-copy-engines-rdma-and-nvswitch-107569.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.38228>)

Author: Zejian Wang, Yifan Zhao, Kaihui Ren, Xiaoling Zhou, Yubin Wang, Chenhao Guan, Lin Zhang, Yufeng Li, Zhikang Mo, Zhuoru Ma, Yingtong Xiong, Jihu Guo, Yiqiao Liao, Chengru Song

Published: 2026-10-30T04:00:00Z

Content type: article

Language: en

Sources: [arXiv: Distributed Computing](<https://devfeed.tech/sources/arxiv-distributed-computing.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [rdma](<https://devfeed.tech/topics/rdma.md>), [Distributed Systems & Parallel Computing](<https://devfeed.tech/topics/distributed-systems-parallel-computing.md>), [NCCL](<https://devfeed.tech/topics/nccl.md>)

Tags: [collective](<https://devfeed.tech/tags/collective.md>), [communication](<https://devfeed.tech/tags/communication.md>), [compute](<https://devfeed.tech/tags/compute.md>), [concurrent](<https://devfeed.tech/tags/concurrent.md>), [conversion](<https://devfeed.tech/tags/conversion.md>), [copy](<https://devfeed.tech/tags/copy.md>), [cross](<https://devfeed.tech/tags/cross.md>), [cs-dc](<https://devfeed.tech/tags/cs-dc.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-pf](<https://devfeed.tech/tags/cs-pf.md>), [data](<https://devfeed.tech/tags/data.md>), [engines](<https://devfeed.tech/tags/engines.md>), [gradient](<https://devfeed.tech/tags/gradient.md>), [kernels](<https://devfeed.tech/tags/kernels.md>), [layout](<https://devfeed.tech/tags/layout.md>), [layouts](<https://devfeed.tech/tags/layouts.md>), [llama](<https://devfeed.tech/tags/llama.md>), [magnitude](<https://devfeed.tech/tags/magnitude.md>), [memory](<https://devfeed.tech/tags/memory.md>), [multicast](<https://devfeed.tech/tags/multicast.md>), [nccl](<https://devfeed.tech/tags/nccl.md>), [nvswitch](<https://devfeed.tech/tags/nvswitch.md>), [rdma](<https://devfeed.tech/tags/rdma.md>)

### AI overview

TIDE targets two FSDP2 costs: copying between per-parameter tensors and flat collective buffers, and communication kernels competing with compute. It folds placement into collective addressing while preserving FSDP2 storage and compute kernels. TIDE-AG writes shards directly to parameter storage using NVSwitch multicast and paced recursive-doubling RDMA without using SMs for transport or placement. TIDE-RS reads per-parameter bf16 gradients and produces fp32 shards through copy engines, short-lived CTAs for node-local accumulation, and NVSwitch for cross-node reduction; its two-phase pipeline separates gradient reads from network progress, with optional Slot-Reuse. In reported 64-GPU experiments across Llama-3.1 70B, Wan2.2, and Qwen3-30B-A3B, TIDE cut step time by up to 27.1% versus Stock FSDP2 and 6.0% versus FSDP2+SymmMem. The abstract also reports 12.5-14.5x lower reduce-scatter SM time and up to 4.7 GiB less peak memory than FSDP2+SymmMem. These are reported maxima; the supplied evidence is an abstract, not full paper text.

### Source excerpt

arXiv:2610.38228v1 Announce Type: new Abstract: Fully Sharded Data Parallelism (FSDP2) maintains parameters and gradients as per-parameter tensors, whereas its collective path operates on flat, rank-major buffers. Converting between these layouts copies all-gather outputs into parameter storage and packs gradients into full-group fp32 buffers. Layout conversion alone accounts for 6.4% of step time in Llama-3.1 70B training at 8K tokens per GPU. Communication kernels impose a second cost: a default NCCL reduce-scatter can slow a concurrent GEMM that would otherwise complete in a single wave by 69%. We present TIDE, a set of zero-copy collectives built on public NCCL 2.31+ APIs. TIDE preserves FSDP2's per-parameter storage and compute kernels while folding layout placement into collective addressing. TIDE-AG delivers shards directly to parameter storage using NVSwitch multicast and paced recursive-doubling RDMA, with no SMs used for transport or placement. TIDE-RS consumes per-parameter bf16 gradients directly and reduces them into fp32 shards (a mixed-precision mode that NCCL does not provide) using copy engines and short-lived thread blocks (CTAs) for node-local accumulation and NVSwitch for cross-node reduction. A two-phase pipeline separates gradient reads from network progress; Slot-Reuse optionally stages gradients in vacated all-gather slots. On 64 GPUs, TIDE reduces step time by up to 27.1% relative to Stock FSDP2 and 6.0% relative to FSDP2+SymmMem across Llama-3.1 70B, Wan2.2, and Qwen3-30B-A3B. It reduces reduce-scatter SM time by factors of 12.5-14.5 and cuts interference with concurrent compute by up to an order of magnitude. Its reduce-scatter buffer remains nearly constant from 16 to 64 GPUs at a fixed per-GPU output size, and its peak memory is up to 4.7 GiB below FSDP2+SymmMem's and, with Slot-Reuse on Wan2.2, below Stock FSDP2's.

## Auto repair for Together GPU Clusters: from GPU fault to healthy node

DevFeed: [Auto repair for Together GPU Clusters: from GPU fault to healthy node](<https://devfeed.tech/articles/auto-repair-for-together-gpu-clusters-from-gpu-fault-to-healthy-node-103670.md>)

Original publisher: [Read original article](<https://www.together.ai/blog/auto-repair-for-together-gpu-clusters>)

Author: Pavneet Ahluwalia, Dongsheng Zhang, Barbara Kazanowska

Published: 2026-10-27T00:00:00Z

Content type: article

Language: en

Sources: [Together.ai](<https://devfeed.tech/sources/together-ai.md>)

Topics: [Distributed Systems & Parallel Computing](<https://devfeed.tech/topics/distributed-systems-parallel-computing.md>), [NCCL](<https://devfeed.tech/topics/nccl.md>)

Tags: [2](<https://devfeed.tech/tags/2.md>), [3](<https://devfeed.tech/tags/3.md>), [account](<https://devfeed.tech/tags/account.md>), [actions](<https://devfeed.tech/tags/actions.md>), [alert](<https://devfeed.tech/tags/alert.md>), [alerts](<https://devfeed.tech/tags/alerts.md>), [also](<https://devfeed.tech/tags/also.md>), [annotation](<https://devfeed.tech/tags/annotation.md>), [api](<https://devfeed.tech/tags/api.md>), [api-key](<https://devfeed.tech/tags/api-key.md>), [automatically](<https://devfeed.tech/tags/automatically.md>), [clusters](<https://devfeed.tech/tags/clusters.md>), [code](<https://devfeed.tech/tags/code.md>), [fabric](<https://devfeed.tech/tags/fabric.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [gpu-clusters](<https://devfeed.tech/tags/gpu-clusters.md>), [node](<https://devfeed.tech/tags/node.md>), [repair](<https://devfeed.tech/tags/repair.md>), [run](<https://devfeed.tech/tags/run.md>), [works](<https://devfeed.tech/tags/works.md>)

### AI overview

Together GPU Clusters' auto-repair pipeline uses failed health checks and supporting evidence to recommend the lightest mapped fix; `NotReady` alone never triggers repair. Detection runs inside VMs and on hosts, while policy controls which faults run automatically and how long repairs wait for workloads to become idle. Repairs proceed through cordon, wait, drain, action and rejoin, with escalation after repeated failures, evidence and capacity gates for migrations, and safeguards against repair storms. Customers can submit repairs through the API or console; node-label remediation contracts are in preview. After migration or removal, the vacated machine is automatically tested and returned to service, ticketed after explicit failure, or restored to its prior state if validation is aborted. Reboots preserve local scratch data; quick reprovisioning loses it. The excerpt also describes host power cycling as a preview feature.

### Source excerpt

Together GPU Clusters auto repair detects GPU, host and fabric faults, picks the lightest fix, and lets you decide per fault and per Xid code which repairs run automatically. Here is how it works.

## 5 Lessons from Building a Multi-plane Network Fabric for Agentic AI

DevFeed: [5 Lessons from Building a Multi-plane Network Fabric for Agentic AI](<https://devfeed.tech/articles/5-lessons-from-building-a-multi-plane-network-fabric-for-agentic-ai-96071.md>)

Original publisher: [Read original article](<https://wf.coreweave.com/blog/5-lessons-from-building-a-multi-plane-network-fabric-for-agentic-ai>)

Author: CoreWeave

Published: 2026-10-22T16:27:39Z

Content type: article

Language: en

Sources: [Coreweave Blog Posts](<https://devfeed.tech/sources/coreweave-blog-posts.md>)

Topics: [AI Networking](<https://devfeed.tech/topics/ai-networking.md>), [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [Agentic AI](<https://devfeed.tech/topics/what-is-agentic-ai.md>), [Multi Agent Systems](<https://devfeed.tech/topics/multi-agent-systems.md>), [NCCL](<https://devfeed.tech/topics/nccl.md>), [Vera Rubin NVL72](<https://devfeed.tech/topics/vera-rubin-nvl72.md>)

Tags: [access-controls](<https://devfeed.tech/tags/access-controls.md>), [access-management](<https://devfeed.tech/tags/access-management.md>), [adaptive](<https://devfeed.tech/tags/adaptive.md>), [agentic](<https://devfeed.tech/tags/agentic.md>), [agentic-ai](<https://devfeed.tech/tags/agentic-ai.md>), [ai](<https://devfeed.tech/tags/ai.md>), [balancing](<https://devfeed.tech/tags/balancing.md>), [bluefield-3](<https://devfeed.tech/tags/bluefield-3.md>), [bluefield-4](<https://devfeed.tech/tags/bluefield-4.md>), [cks](<https://devfeed.tech/tags/cks.md>), [compliance](<https://devfeed.tech/tags/compliance.md>), [concurrently](<https://devfeed.tech/tags/concurrently.md>), [congestion-control](<https://devfeed.tech/tags/congestion-control.md>), [connectx](<https://devfeed.tech/tags/connectx.md>), [content-type](<https://devfeed.tech/tags/content-type.md>), [data-centers](<https://devfeed.tech/tags/data-centers.md>), [fabric](<https://devfeed.tech/tags/fabric.md>), [inference](<https://devfeed.tech/tags/inference.md>), [network](<https://devfeed.tech/tags/network.md>), [scale](<https://devfeed.tech/tags/scale.md>), [training](<https://devfeed.tech/tags/training.md>)

### AI overview

Agent workloads create unpredictable, serial chains of network hops whose latency adds up, so a fabric must carry them alongside training and inference without mutual degradation. The article describes CoreWeave's approach: 1:1 non-oversubscribed links; Spectrum-X adaptive routing and telemetry-based congestion control; multi-plane and multi-rail designs to scale while keeping paths short; and ConnectX-9 hardware load balancing that reacts to congestion without host CPU scheduling. It also describes resilience across topology, BlueField DPUs, software, operations, and geography, plus flow-level observability and Telemetry Relay. These are the provider's reported design choices and capabilities; the text offers no independent comparative measurements. It notes that congestion controls require workload-specific tuning, since aggressive pausing can cause head-of-line blocking and tail latency.

### Source excerpt

Five lessons we learned from building a multi-plane network that can handle training, inference, and agentic workloads concurrently, at scale.

## Engineered for Agentic AI: NVIDIA HGX B300 on CoreWeave Cloud

DevFeed: [Engineered for Agentic AI: NVIDIA HGX B300 on CoreWeave Cloud](<https://devfeed.tech/articles/engineered-for-agentic-ai-nvidia-hgx-b300-on-coreweave-cloud-96124.md>)

Original publisher: [Read original article](<https://wf.coreweave.com/blog/engineered-for-agentic-ai-nvidia-hgx-b300-on-coreweave-cloud>)

Author: Harsh Singh Banwait

Published: 2026-10-22T16:27:39Z

Content type: release

Language: en

Sources: [Coreweave Blog Posts](<https://devfeed.tech/sources/coreweave-blog-posts.md>)

Topics: [Agentic AI](<https://devfeed.tech/topics/what-is-agentic-ai.md>), [Nvidia](<https://devfeed.tech/topics/nvidia.md>), [NVFP4](<https://devfeed.tech/topics/nvfp4.md>), [NVLink](<https://devfeed.tech/topics/nvlink.md>), [Cloud](<https://devfeed.tech/topics/cloud.md>), [Multi-GPU](<https://devfeed.tech/topics/multi-gpu.md>), [NCCL](<https://devfeed.tech/topics/nccl.md>), [observability](<https://devfeed.tech/topics/observability.md>), [AI Networking](<https://devfeed.tech/topics/ai-networking.md>), [Cloud Architecture](<https://devfeed.tech/topics/cloud-architecture.md>)

Tags: [about](<https://devfeed.tech/tags/about.md>), [accelerate](<https://devfeed.tech/tags/accelerate.md>), [accelerated](<https://devfeed.tech/tags/accelerated.md>), [accelerators](<https://devfeed.tech/tags/accelerators.md>), [advanced](<https://devfeed.tech/tags/advanced.md>), [agent](<https://devfeed.tech/tags/agent.md>), [agent-workflows](<https://devfeed.tech/tags/agent-workflows.md>), [agentic](<https://devfeed.tech/tags/agentic.md>), [agentic-ai](<https://devfeed.tech/tags/agentic-ai.md>), [agentic-ai-development](<https://devfeed.tech/tags/agentic-ai-development.md>), [agentic-ai-systems](<https://devfeed.tech/tags/agentic-ai-systems.md>), [agentic-systems](<https://devfeed.tech/tags/agentic-systems.md>), [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [b300](<https://devfeed.tech/tags/b300.md>), [cloud](<https://devfeed.tech/tags/cloud.md>), [coreweave](<https://devfeed.tech/tags/coreweave.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [performance](<https://devfeed.tech/tags/performance.md>), [scale](<https://devfeed.tech/tags/scale.md>)

### AI overview

CoreWeave says NVIDIA HGX B300 is generally available on its cloud for agentic AI workloads, pairing higher memory and NVFP4 performance with networking, orchestration, monitoring, and liquid cooling. The excerpt reports 270 GB of GPU memory versus 180 GB on HGX B200, and 14 PFLOPs of NVFP4 performance versus 9 PFLOPs; it says high-precision performance remains at the B200 level. CoreWeave's benchmarks, compared with HGX H200, report 3.42x faster token generation on Kimi K2.5, 2.61x on DeepSeek-R1, 4.93x faster DeepSeek-R1 end-to-end latency, and 1.95x measured collective bandwidth. Compared with B200, Kimi K2.5 concurrency improved 1.41x. The tests used different model configurations across architectures, including NVFP4 on B300 versus FP8 block scaling on H200 for DeepSeek-R1. The platform also offers Mission Control performance visibility, Kubernetes-based workload orchestration, Slurm on Kubernetes, W&B Weave tracing, and object storage cited at up to 7 GB/s per GPU.

### Source excerpt

Unlock the performance of NVIDIA HGX B300 at scale on CoreWeave.

## Bitdeer AI Brings NVIDIA Minerva's AI-Native AIOps For Maximizing AI Factory Yield

DevFeed: [Bitdeer AI Brings NVIDIA Minerva's AI-Native AIOps For Maximizing AI Factory Yield](<https://devfeed.tech/articles/bitdeer-ai-brings-nvidia-minerva-s-ai-native-aiops-for-maximizing-ai-factory-yield-95860.md>)

Original publisher: [Read original article](<https://www.bitdeer.ai/en/blog/nvidia-minerva/>)

Author: Evelyn Xiong

Published: 2026-10-22T15:09:48Z

Content type: release

Language: en

Sources: [Bitdeer AI Cloud](<https://devfeed.tech/sources/bitdeer-ai-cloud.md>)

Topics: [AIOps](<https://devfeed.tech/topics/aiops.md>), [AI in IT operations](<https://devfeed.tech/topics/ai-in-it-operations.md>), [NVIDIA DSX](<https://devfeed.tech/topics/nvidia-dsx.md>), [NCCL](<https://devfeed.tech/topics/nccl.md>)

Tags: [advanced](<https://devfeed.tech/tags/advanced.md>), [agent](<https://devfeed.tech/tags/agent.md>), [agentic](<https://devfeed.tech/tags/agentic.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-assistant](<https://devfeed.tech/tags/ai-assistant.md>), [ai-factories](<https://devfeed.tech/tags/ai-factories.md>), [ai-factory](<https://devfeed.tech/tags/ai-factory.md>), [ai-infrastructure](<https://devfeed.tech/tags/ai-infrastructure.md>), [ai-ml](<https://devfeed.tech/tags/ai-ml.md>), [ai-native](<https://devfeed.tech/tags/ai-native.md>), [ai-training](<https://devfeed.tech/tags/ai-training.md>), [ai-training-and-inference](<https://devfeed.tech/tags/ai-training-and-inference.md>), [ai-workloads](<https://devfeed.tech/tags/ai-workloads.md>), [aiops](<https://devfeed.tech/tags/aiops.md>), [alert](<https://devfeed.tech/tags/alert.md>), [bitdeer](<https://devfeed.tech/tags/bitdeer.md>), [factory](<https://devfeed.tech/tags/factory.md>), [native](<https://devfeed.tech/tags/native.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [yield](<https://devfeed.tech/tags/yield.md>)

### AI overview

Bitdeer AI says it now supports NVIDIA Minerva, an AI-native AIOps platform for AI Factory infrastructure. Minerva correlates metric, log, and topology telemetry across NVIDIA hardware, networking, NCCL, schedulers, and AI workloads into a cluster-and-job knowledge graph. It analyzes telemetry in real time to learn normal behavior, predict anomalies, and correlate events; operators can use its MCP server and Sherlock assistant to investigate live telemetry in natural language. NVIDIA reports internal test results including one NVMe power-fault repair reduced from a multi-day outage to six hours, and 81 noisy events consolidated into one alert, saving 54 hours of cluster uptime. Bitdeer says the platform is available standalone or integrated with NVIDIA DSX OS and is intended to improve uptime and productive GPU hours as its infrastructure scales.

### Source excerpt

Bitdeer AI Brings NVIDIA Minerva's AI-Native AIOps For Maximizing AI Factory Yield.

## vLLM Support for NVIDIA Vera Rubin NVL72: 7.8x Throughput over GB200 NVL72

DevFeed: [vLLM Support for NVIDIA Vera Rubin NVL72: 7.8x Throughput over GB200 NVL72](<https://devfeed.tech/articles/vllm-support-for-nvidia-vera-rubin-nvl72-7-8x-throughput-over-gb200-nvl72-84458.md>)

Original publisher: [Read original article](<https://vllm.ai/blog/2026-10-09-vera-rubin-preview>)

Author: vLLM Team, Inferact, Red Hat, and NVIDIA

Published: 2026-10-09T00:00:00Z

Content type: article

Language: en

Sources: [vLLM Blog](<https://devfeed.tech/sources/vllm-blog.md>)

Topics: [vllm](<https://devfeed.tech/topics/vllm.md>), [NVIDIA Vera Rubin](<https://devfeed.tech/topics/nvidia-vera-rubin.md>), [Vera Rubin NVL72](<https://devfeed.tech/topics/vera-rubin-nvl72.md>), [performance-engineering](<https://devfeed.tech/topics/performance-engineering.md>), [NCCL](<https://devfeed.tech/topics/nccl.md>), [NVLink](<https://devfeed.tech/topics/nvlink.md>)

Tags: [ai-engineering](<https://devfeed.tech/tags/ai-engineering.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [hardware](<https://devfeed.tech/tags/hardware.md>), [llm-inference](<https://devfeed.tech/tags/llm-inference.md>), [locality](<https://devfeed.tech/tags/locality.md>), [model-serving](<https://devfeed.tech/tags/model-serving.md>), [moe](<https://devfeed.tech/tags/moe.md>), [nvidia-vera](<https://devfeed.tech/tags/nvidia-vera.md>), [open-source-ai](<https://devfeed.tech/tags/open-source-ai.md>), [performance](<https://devfeed.tech/tags/performance.md>), [vera-rubin-nvl72](<https://devfeed.tech/tags/vera-rubin-nvl72.md>), [vllm](<https://devfeed.tech/tags/vllm.md>), [vllm-blog](<https://devfeed.tech/tags/vllm-blog.md>)

### AI overview

vLLM now runs on NVIDIA Vera Rubin NVL72, with daily container builds, support for several models, Rubin-tuned kernels, and locality-aware MoE optimizations. Early benchmarks report up to 7.84 times the per-GPU throughput of GB200 NVL72 on AgentX at matched interactivity, and up to 3.7 times the throughput of GB300 NVL72 on an MLPerf vision-language benchmark.

### Source excerpt

vLLM now runs on NVIDIA Vera Rubin NVL72 with day-0 model support, Rubin-tuned FlashInfer kernels, and locality-aware MoE, delivering 7.8x the per-GPU throughput of GB200 NVL72 on AgentX.

## How DOCA GPUNetIO Unifies GPU-Initiated Networking Across the NVIDIA Software Stack

DevFeed: [How DOCA GPUNetIO Unifies GPU-Initiated Networking Across the NVIDIA Software Stack](<https://devfeed.tech/articles/how-doca-gpunetio-unifies-gpu-initiated-networking-across-the-nvidia-software-stack-65883.md>)

Original publisher: [Read original article](<https://developer.nvidia.com/blog/doca-gpunetio-gda-ki-unified-gpu-networking/>)

Author: Elizabeth Goodman

Published: 2026-10-06T19:07:02Z

Content type: article

Language: en

Sources: [NVIDIA Developer](<https://devfeed.tech/sources/nvidia-developer.md>), [NVIDIA Technical Blog](<https://devfeed.tech/sources/nvidia-technical-blog.md>)

Topics: [gpudirect rdma](<https://devfeed.tech/topics/gpudirect-rdma.md>), [DOCA](<https://devfeed.tech/topics/doca.md>), [NCCL](<https://devfeed.tech/topics/nccl.md>), [Multi-GPU](<https://devfeed.tech/topics/multi-gpu.md>), [GPUDirect](<https://devfeed.tech/topics/gpudirect.md>), [scalable-architecture](<https://devfeed.tech/topics/scalable-architecture.md>)

Tags: [architecture](<https://devfeed.tech/tags/architecture.md>), [data-center-cloud](<https://devfeed.tech/tags/data-center-cloud.md>), [doca](<https://devfeed.tech/tags/doca.md>), [edge-computing](<https://devfeed.tech/tags/edge-computing.md>), [featured](<https://devfeed.tech/tags/featured.md>), [nccl](<https://devfeed.tech/tags/nccl.md>), [networking](<https://devfeed.tech/tags/networking.md>), [networking-communications](<https://devfeed.tech/tags/networking-communications.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [nvlink](<https://devfeed.tech/tags/nvlink.md>), [nvshmem](<https://devfeed.tech/tags/nvshmem.md>), [software](<https://devfeed.tech/tags/software.md>)

### AI overview

NVIDIA's DOCA GPUNetIO provides a shared foundation for GPU-initiated networking, allowing CUDA kernels to drive Ethernet and RDMA operations while reducing CPU involvement in the application data path. The article describes its open-source and DOCA SDK implementations, programming model, and integrations with libraries including NCCL and NVSHMEM. It reports that enabling GDA-KI in an NVSHMEM benchmark improved scaling across CTAs and QPs for small messages by removing a CPU proxy bottleneck.

### Source excerpt

GPU applications increasingly need networking and data movement to behave like first-class GPU-controlled operations rather than host-driven services. When the...

## 755GB to 395GB: How Bitdeer AI Halves GLM-5.3's GPU Footprint

DevFeed: [755GB to 395GB: How Bitdeer AI Halves GLM-5.3's GPU Footprint](<https://devfeed.tech/articles/755gb-to-395gb-how-bitdeer-ai-halves-glm-5-3-s-gpu-footprint-87579.md>)

Original publisher: [Read original article](<https://www.bitdeer.ai/en/blog/755gb-to-395gb-how-bitdeer-ai-halves-glm-5-3s-gpu-footprint/>)

Author: Evelyn Xiong

Published: 2026-10-05T08:33:27Z

Content type: article

Language: en

Sources: [Bitdeer AI Cloud](<https://devfeed.tech/sources/bitdeer-ai-cloud.md>)

Topics: [Mixture of Experts (MoE)](<https://devfeed.tech/topics/mixture-of-experts-moe.md>), [quantization](<https://devfeed.tech/topics/quantization.md>), [NCCL](<https://devfeed.tech/topics/nccl.md>), [baseten](<https://devfeed.tech/topics/baseten.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [ai-applications](<https://devfeed.tech/tags/ai-applications.md>), [artificial-analysis](<https://devfeed.tech/tags/artificial-analysis.md>), [awq](<https://devfeed.tech/tags/awq.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [deepseek](<https://devfeed.tech/tags/deepseek.md>), [deepseek-v3](<https://devfeed.tech/tags/deepseek-v3.md>), [experts](<https://devfeed.tech/tags/experts.md>), [fp8](<https://devfeed.tech/tags/fp8.md>), [glm](<https://devfeed.tech/tags/glm.md>), [glm-5-3](<https://devfeed.tech/tags/glm-5-3.md>), [gpu](<https://devfeed.tech/tags/gpu.md>)

### AI overview

Bitdeer AI describes quantizing GLM-5.3 to W4AFP8, reducing its checkpoint from 755 GB to about 395 GB so it can serve on one 8-GPU node. Its expert-parallel GPTQ pipeline reduces calibration memory and checkpoint build time compared with AWQ. The reported evaluations found similar scores across the compared quantized checkpoints, while expert-parallel GPTQ used fewer output tokens on two benchmarks.

### Source excerpt

We quantized GLM-5.3, down to 4-bit experts with FP8 activations. The result is a checkpoint of about 395 GB that serves on a single node.

## Modal Clusters are generally available

DevFeed: [Modal Clusters are generally available](<https://devfeed.tech/articles/modal-clusters-are-generally-available-79886.md>)

Original publisher: [Read original article](<https://modal.com/blog/modal-clusters-generally-available>)

Author: Peyton Walters

Published: 2026-10-01T12:00:00Z

Content type: release

Language: en

Sources: [Modal](<https://devfeed.tech/sources/modal.md>)

Topics: [Distributed Systems & Parallel Computing](<https://devfeed.tech/topics/distributed-systems-parallel-computing.md>), [rdma](<https://devfeed.tech/topics/rdma.md>), [Multi-GPU](<https://devfeed.tech/topics/multi-gpu.md>), [NCCL](<https://devfeed.tech/topics/nccl.md>)

Tags: [autoscaling](<https://devfeed.tech/tags/autoscaling.md>), [containers](<https://devfeed.tech/tags/containers.md>), [gpu-clusters](<https://devfeed.tech/tags/gpu-clusters.md>), [rdma](<https://devfeed.tech/tags/rdma.md>)

### AI overview

Modal announces general availability of Modal Clusters, a multi-node GPU compute offering scheduled from shared capacity and billed by the second. It describes gang scheduling, RDMA support, GPU health remediation, and use cases for AI training and inference.

### Source excerpt

Multi-node GPU clusters with RDMA, gang scheduled from Modal's shared capacity pool and billed by the second, behind a single decorator.

## Validate GPU Cluster Readiness Before AI Workloads Land

DevFeed: [Validate GPU Cluster Readiness Before AI Workloads Land](<https://devfeed.tech/articles/validate-gpu-cluster-readiness-before-ai-workloads-land-58917.md>)

Original publisher: [Read original article](<https://developer.nvidia.com/blog/validate-gpu-cluster-readiness-before-ai-workloads-land/>)

Author: Michelle Horton

Published: 2026-09-23T19:45:19Z

Content type: article

Language: en

Sources: [NVIDIA Developer](<https://devfeed.tech/sources/nvidia-developer.md>), [NVIDIA Technical Blog](<https://devfeed.tech/sources/nvidia-technical-blog.md>)

Topics: [AI Platform](<https://devfeed.tech/topics/ai-platform.md>), [Kubernetes](<https://devfeed.tech/topics/kubernetes.md>), [GPU](<https://devfeed.tech/topics/gpu.md>), [NCCL](<https://devfeed.tech/topics/nccl.md>), [Orchestration](<https://devfeed.tech/topics/orchestration.md>), [rdma](<https://devfeed.tech/topics/rdma.md>), [GitOps](<https://devfeed.tech/topics/gitops.md>), [Open Source](<https://devfeed.tech/topics/open-source.md>)

Tags: [ai-factory](<https://devfeed.tech/tags/ai-factory.md>), [ai-platforms-deployment](<https://devfeed.tech/tags/ai-platforms-deployment.md>), [cluster](<https://devfeed.tech/tags/cluster.md>), [data-center-cloud](<https://devfeed.tech/tags/data-center-cloud.md>), [dsx](<https://devfeed.tech/tags/dsx.md>), [featured](<https://devfeed.tech/tags/featured.md>), [gitops](<https://devfeed.tech/tags/gitops.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [hardware](<https://devfeed.tech/tags/hardware.md>), [kubernetes](<https://devfeed.tech/tags/kubernetes.md>), [nccl](<https://devfeed.tech/tags/nccl.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [production](<https://devfeed.tech/tags/production.md>), [rdma](<https://devfeed.tech/tags/rdma.md>)

### AI overview

NVIDIA presents NVIDIA Cluster Readiness Engine, an open-source Kubernetes controller for validating GPU cluster readiness before production AI workloads run. It uses topology-aware distributed workloads to identify failing nodes and supports staged readiness testing through Kubernetes resources and GitOps workflows.

### Source excerpt

A GPU cluster can pass every health check and still fail to run an AI workload. Even when every GPU, network link, and pod reports healthy, a 512-GPU training...

[Next page](<https://devfeed.tech/topics/nccl.md?cursor=WyIyMDI2LTA5LTIzVDE5OjQ1OjE5KzAwOjAwIiwgIjMzODU3NTkwLTU0NTYtNGQ4OS04YjVlLWNmN2FkODk2NzkwMSJd>)