# rdma

Remote Direct Memory Access (RDMA) is a technology that enables network adapters to steer data directly into and out of application buffers, supporting direct memory access between peers.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## RoCE and NVIDIA Quantum InfiniBand, and their quantified tradeoffs

DevFeed: [RoCE and NVIDIA Quantum InfiniBand, and their quantified tradeoffs](<https://devfeed.tech/articles/roce-and-nvidia-quantum-infiniband-and-their-quantified-tradeoffs-146467.md>)

Original publisher: [Read original article](<https://lambda.com/blog/roce-and-nvidia-quantum-infiniband-and-their-quantified-tradeoffs>)

Author: Zach Mueller

Published: 2026-11-20T14:08:48Z

Content type: comparison

Language: en

Sources: [The Lambda deep learning blog](<https://devfeed.tech/sources/the-lambda-deep-learning-blog.md>)

Topics: [InfiniBand](<https://devfeed.tech/topics/infiniband.md>), [rdma](<https://devfeed.tech/topics/rdma.md>), [Distributed Systems & Parallel Computing](<https://devfeed.tech/topics/distributed-systems-parallel-computing.md>), [NCCL](<https://devfeed.tech/topics/nccl.md>), [Nvidia](<https://devfeed.tech/topics/nvidia.md>)

Tags: [1-click-cluster](<https://devfeed.tech/tags/1-click-cluster.md>), [cluster](<https://devfeed.tech/tags/cluster.md>), [infiniband](<https://devfeed.tech/tags/infiniband.md>), [llm-inference](<https://devfeed.tech/tags/llm-inference.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [quantum](<https://devfeed.tech/tags/quantum.md>), [roce](<https://devfeed.tech/tags/roce.md>), [tradeoffs](<https://devfeed.tech/tags/tradeoffs.md>)

### AI overview

RoCEv2 uses Ethernet with UDP/IP and can span subnets, while NVIDIA Quantum InfiniBand uses a dedicated fabric that requires InfiniBand switches and infrastructure. The text gives typical latency ranges of 0.5-2 µs for InfiniBand and 1-3 µs for optimized local RoCE, noting RoCE is more sensitive to configuration, congestion, and topology. In a two-node HGX B200 inference test using GLM-5.2-NVFP4 and SGLang, RoCE showed roughly 70% lower throughput than InfiniBand at about 250 concurrent users and 2.5x slower time to first token. The authors caution that real-world performance varies and recommend tempering expectations for On-Demand nodes; the comparison describes one setup.

### Source excerpt

The GPUs you rent are just part of what determines your cluster's speed and efficiency. This comes into play through interconnect, the network that connects multiple nodes (machines) on the backend.

## November 2026 Top500 New Systems Are Few and Far Between

DevFeed: [November 2026 Top500 New Systems Are Few and Far Between](<https://devfeed.tech/articles/november-2026-top500-new-systems-are-few-and-far-between-145836.md>)

Original publisher: [Read original article](<https://www.servethehome.com/november-2026-top500-new-systems-analysis/>)

Author: Patrick Kennedy

Published: 2026-11-19T17:48:48Z

Content type: news

Language: en

Sources: [ServeTheHome](<https://devfeed.tech/sources/servethehome.md>)

Topics: [High-Performance Computing](<https://devfeed.tech/topics/high-performance-computing.md>), [systems](<https://devfeed.tech/topics/systems.md>), [rdma](<https://devfeed.tech/topics/rdma.md>)

Tags: [17](<https://devfeed.tech/tags/17.md>), [2021](<https://devfeed.tech/tags/2021.md>), [2022](<https://devfeed.tech/tags/2022.md>), [2023](<https://devfeed.tech/tags/2023.md>), [2026](<https://devfeed.tech/tags/2026.md>), [ai](<https://devfeed.tech/tags/ai.md>), [amd](<https://devfeed.tech/tags/amd.md>), [amd-epyc](<https://devfeed.tech/tags/amd-epyc.md>), [ampere](<https://devfeed.tech/tags/ampere.md>), [arm](<https://devfeed.tech/tags/arm.md>), [blackwell](<https://devfeed.tech/tags/blackwell.md>), [china](<https://devfeed.tech/tags/china.md>), [china-telecom](<https://devfeed.tech/tags/china-telecom.md>), [churn](<https://devfeed.tech/tags/churn.md>), [cloud](<https://devfeed.tech/tags/cloud.md>), [cluster](<https://devfeed.tech/tags/cluster.md>), [clusters](<https://devfeed.tech/tags/clusters.md>), [intel](<https://devfeed.tech/tags/intel.md>), [news](<https://devfeed.tech/tags/news.md>), [november-2026](<https://devfeed.tech/tags/november-2026.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [systems](<https://devfeed.tech/tags/systems.md>), [top500](<https://devfeed.tech/tags/top500.md>), [trends](<https://devfeed.tech/tags/trends.md>)

### AI overview

The November 2026 TOP500 added just 28 systems--the lowest count in the article's tracking series since June 2018--and total list performance grew about 5%. Only five newcomers reached the Top 100; none made the Top 10. The average listed system is now about 3.2 years old. Twenty-three new entries use accelerators, including 14 with NVIDIA Blackwell, while all five CPU-only newcomers use AMD EPYC. China contributed one new system and has 18 systems on the list, down from 173 in November 2021; only two newcomers are classified as Industry. InfiniBand appears in 19 of the 28 new systems. The authors caution that TOP500 submissions reflect declared HPL configurations, not a census of HPC or AI deployments.

### Source excerpt

We check out the new systems in the November 2026 Top500 list and see some of the major trends shaking out The post November 2026 Top500 New Systems Are Few and Far Between appeared first on ServeTheHome.

## GNoF: A Low-SM-Overhead GPU-Native NVMe-over-Fabrics Completion Path for Data-Intensive GPU Workloads

DevFeed: [GNoF: A Low-SM-Overhead GPU-Native NVMe-over-Fabrics Completion Path for Data-Intensive GPU Workloads](<https://devfeed.tech/articles/gnof-a-low-sm-overhead-gpu-native-nvme-over-fabrics-completion-path-for-data-intensive-gpu-workloads-136997.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.21288>)

Author: Lei Tian, Wenhao Gu

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Distributed Computing](<https://devfeed.tech/sources/arxiv-distributed-computing.md>)

Topics: [gpu-initated](<https://devfeed.tech/topics/gpu-initated.md>), [NVMe](<https://devfeed.tech/topics/nvme.md>), [rdma](<https://devfeed.tech/topics/rdma.md>), [Cache](<https://devfeed.tech/topics/cache.md>)

Tags: [bluefield-3](<https://devfeed.tech/tags/bluefield-3.md>), [cpu](<https://devfeed.tech/tags/cpu.md>), [cs-ar](<https://devfeed.tech/tags/cs-ar.md>), [cs-dc](<https://devfeed.tech/tags/cs-dc.md>), [doorbell](<https://devfeed.tech/tags/doorbell.md>), [dpu](<https://devfeed.tech/tags/dpu.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [infiniband](<https://devfeed.tech/tags/infiniband.md>), [lock-free](<https://devfeed.tech/tags/lock-free.md>), [nvme](<https://devfeed.tech/tags/nvme.md>), [nvme-of](<https://devfeed.tech/tags/nvme-of.md>), [page-cache](<https://devfeed.tech/tags/page-cache.md>), [pipelining](<https://devfeed.tech/tags/pipelining.md>), [rdma](<https://devfeed.tech/tags/rdma.md>), [spdk](<https://devfeed.tech/tags/spdk.md>), [staging](<https://devfeed.tech/tags/staging.md>), [storage](<https://devfeed.tech/tags/storage.md>)

### AI overview

GNoF moves NVMe-over-Fabrics/RDMA command submission and completion handling onto GPU threads, leaving the CPU to set up connections, manage the admin queue, and keep the connection alive. To reduce wasted GPU occupancy over long RDMA round trips, it coalesces doorbells, batches completion-queue polling with wakeup-safe counters, and uses a pinned GPU page cache with pipelined fetch and wait. It requires no target or firmware changes and runs with an unmodified SPDK target. On 200 Gb/s InfiniBand, the authors report 1.77 million 4 KB IOPS per client at 0.37% host CPU and about 1% SM occupancy, compared with 40% CPU for kernel nvme-rdma and about 100% SM occupancy for lock-step polling. Under the tested identical workloads, they also report 10-22x gains on selective column I/O against three baselines; the abstract notes a regime where bulk staging is faster.

### Source excerpt

arXiv:2611.21288v1 Announce Type: new Abstract: GPUs are the main compute engine for data-intensive workloads whose datasets exceed device memory and reside on remote flash. The conventional host CPU orchestrates every NVMe-over-Fabrics (NVMe-oF) transaction and stages data through host memory, costing high CPU usage, repeated CPU-GPU synchronization, and redundant data movement. GPU-initiated storage removes the CPU from the I/O path, but existing designs leave two problems: synchronous lock-step completion pins streaming multiprocessors (SMs) in spin-polling, wasteful over a 10-100 microsecond RDMA round trip, and current GPU-native remote systems focus on target-side firmware while using per-command doorbells and periodic CQ polling. We present GNoF, a GPU-native NVMe-oF/RDMA data path where GPU threads build command capsules, post send work-queue elements, ring doorbells, and reap completions entirely on the device; the CPU only handles one-time connection setup, the admin queue, and a keep-alive thread. Three mechanisms target long fabric round trips: a doorbell mover that coalesces a burst's doorbells into one MMIO ring; a batched CQ poller with a lock-free fast peek and monotonic completion counters that cannot lose a wakeup; and a set-associative, reference-pinned GPU page cache with split fetch/wait pipelining and orphan-fill takeover. GNoF needs no target or firmware change and runs against an unmodified SPDK target. On a 200 Gb/s InfiniBand fabric, GNoF reaches 1.77 million 4 KB IOPS per client at 0.37% host CPU and about 1% SM occupancy, versus 40% CPU for kernel nvme-rdma and about 100% SM occupancy for lock-step polling. Against three existing baselines (kernel nvme-rdma, BaM over a BlueField-3 DPU, and BaM on a local SSD) under identical workloads, we show 10-22x gains on selective column I/O, CPU-burn immunity, and the regime where bulk staging is faster.

## Zero-SM NCCL: A Systematic Guide to Designing and Deploying SM-Free GPU Communication

DevFeed: [Zero-SM NCCL: A Systematic Guide to Designing and Deploying SM-Free GPU Communication](<https://devfeed.tech/articles/zero-sm-nccl-a-systematic-guide-to-designing-and-deploying-sm-free-gpu-communication-134894.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.18075>)

Author: Zhenhao He, Siyuan Shen, Ke Wen, Prajwal Singhania, Shiqing Fan, Youngeun Kwon, Sangkug Lym, Vasudevan Rengasamy, Vishnuswaroop Ramesh, Yicheng Qiang, Craig Belusar, Jeff Hammond, Torsten Hoefler

Published: 2026-11-16T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Distributed Computing](<https://devfeed.tech/sources/arxiv-distributed-computing.md>)

Topics: [NCCL](<https://devfeed.tech/topics/nccl.md>), [rdma](<https://devfeed.tech/topics/rdma.md>), [Multi-GPU](<https://devfeed.tech/topics/multi-gpu.md>), [Distributed Systems & Parallel Computing](<https://devfeed.tech/topics/distributed-systems-parallel-computing.md>), [NVLink](<https://devfeed.tech/topics/nvlink.md>), [GPU](<https://devfeed.tech/topics/gpu.md>)

Tags: [accelerate](<https://devfeed.tech/tags/accelerate.md>), [apis](<https://devfeed.tech/tags/apis.md>), [bandwidth](<https://devfeed.tech/tags/bandwidth.md>), [building](<https://devfeed.tech/tags/building.md>), [communication](<https://devfeed.tech/tags/communication.md>), [copy](<https://devfeed.tech/tags/copy.md>), [cpu](<https://devfeed.tech/tags/cpu.md>), [cs-dc](<https://devfeed.tech/tags/cs-dc.md>), [cuda](<https://devfeed.tech/tags/cuda.md>), [data](<https://devfeed.tech/tags/data.md>), [data-movement](<https://devfeed.tech/tags/data-movement.md>), [deep](<https://devfeed.tech/tags/deep.md>), [designing](<https://devfeed.tech/tags/designing.md>), [domain](<https://devfeed.tech/tags/domain.md>), [domains](<https://devfeed.tech/tags/domains.md>), [free](<https://devfeed.tech/tags/free.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [guide](<https://devfeed.tech/tags/guide.md>), [nccl](<https://devfeed.tech/tags/nccl.md>), [sm](<https://devfeed.tech/tags/sm.md>), [zero](<https://devfeed.tech/tags/zero.md>)

### AI overview

Zero-SM NCCL avoids SM kernels by synchronizing with CUDA stream memory operations, using copy engines for transfers within a scale-up domain, and driving network RDMA through a stream-ordered CPU proxy. It supports AllGather and AlltoAll, plus one-sided ncclPutSignal and ncclWaitSignal APIs. On GB200 NVL72 and B200 NVL8, collectives achieved up to 1.36x NCCL's legacy AllGather bandwidth and reduced median overlapped GEMM slowdown from 33% to 4%, with higher per-operation latency. In Megatron-LM, FSDP improved by up to 3.5% and pipeline parallelism by up to 15% with one-sided puts. The gains depend on workload: the approach benefits large transfers in GPU-bound workloads and deep pipelines with large bubbles, but loses on small messages and host-bound workloads.

### Source excerpt

arXiv:2611.18075v1 Announce Type: new Abstract: GPU communication libraries such as NCCL use Streaming Multiprocessors (SMs) for data movement and synchronization, slowing overlapped computation at a cost that grows with interconnect bandwidth. We present Zero-SM NCCL, which communicates within and across scale-up domains without launching any SM kernel. It synchronizes through CUDA stream memory operations, moves data asynchronously with copy engines within a scale-up domain, and drives network RDMA from a stream-ordered CPU proxy. It supports data-movement collectives such as AllGather and AlltoAll, plus one-sided ncclPutSignal and ncclWaitSignal APIs. On GB200 NVL72 and B200 NVL8 systems, Zero-SM collectives reach up to 1.36x higher AllGather bandwidth than NCCL's legacy SM-based kernels and cut the median slowdown of overlapped GEMMs from 33% to 4%, at a higher latency per operation. In Megatron-LM training, they accelerate FSDP by up to 3.5%, and one-sided puts accelerate pipeline parallelism by up to 15%. Zero-SM communication is not a universal win, however. Through an end-to-end study across models and parallelism strategies, we show that it pays off for large transfers in GPU-bound workloads and for deep pipelines with large bubbles, but loses on small messages and host-bound workloads. We distill these findings into guidelines for enabling zero-SM communication in NCCL and building it into other libraries.

## Modal becomes a gVisor maintainer

DevFeed: [Modal becomes a gVisor maintainer](<https://devfeed.tech/articles/modal-becomes-a-gvisor-maintainer-129819.md>)

Original publisher: [Read original article](<https://modal.com/blog/modal-gvisor-maintainer>)

Author: Luis Capelo

Published: 2026-11-11T12:00:00Z

Content type: article

Language: en

Sources: [Modal](<https://devfeed.tech/sources/modal.md>)

Topics: [virtualization](<https://devfeed.tech/topics/virtualization.md>), [gpudirect rdma](<https://devfeed.tech/topics/gpudirect-rdma.md>), [rdma](<https://devfeed.tech/topics/rdma.md>), [cgroups](<https://devfeed.tech/topics/cgroups.md>), [GPU rendering](<https://devfeed.tech/topics/gpu-rendering.md>), [Cloud Gaming](<https://devfeed.tech/topics/cloud-gaming.md>)

Tags: [agentic](<https://devfeed.tech/tags/agentic.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-coding-agents](<https://devfeed.tech/tags/ai-coding-agents.md>), [amazon-s3](<https://devfeed.tech/tags/amazon-s3.md>), [aws](<https://devfeed.tech/tags/aws.md>), [bare-metal](<https://devfeed.tech/tags/bare-metal.md>), [blackwell](<https://devfeed.tech/tags/blackwell.md>), [blackwell-gpu](<https://devfeed.tech/tags/blackwell-gpu.md>), [clickhouse](<https://devfeed.tech/tags/clickhouse.md>), [cloud](<https://devfeed.tech/tags/cloud.md>), [cloud-gaming](<https://devfeed.tech/tags/cloud-gaming.md>), [cloud-native-computing-foundation](<https://devfeed.tech/tags/cloud-native-computing-foundation.md>), [clusters](<https://devfeed.tech/tags/clusters.md>), [containers](<https://devfeed.tech/tags/containers.md>), [gvisor](<https://devfeed.tech/tags/gvisor.md>), [modal](<https://devfeed.tech/tags/modal.md>)

### AI overview

Modal is joining gVisor's maintainers as the project moves to CNCF governance, building on its use of gVisor for more than 40 million containers a day. gVisor places a Go-based application kernel (the Sentry) between container workloads and the host kernel, then layers Linux isolation controls around it; unlike hardware virtualization, it can run on Linux machines without nested virtualization. Modal cites multi-tenant isolation, GPU access through nvproxy, roughly 25 ms startup with 12 MB memory overhead, checkpoint and restore, and portability across clouds and bare-metal systems as reasons for choosing it. Its upstream work includes GPU driver and workload support, an RDMA proxy for multi-node training, snapshots, and filesystem and shared-memory performance improvements. The article reports about 99% of native bus bandwidth in NCCL all-reduce benchmarks for RDMA and describes multi-GPU snapshots and GPUDirect Storage as ongoing work. Modal's vision is broader accelerator support, portable state, and faster I/O under neutral, multi-vendor governance.

### Source excerpt

Modal runs upwards of 40 million gVisor containers every day. Here's why we bet on it, what we've built into it, and where we want to take it next.

## fRAC: Remote Accelerator Calls on Hostless FPGAs

DevFeed: [fRAC: Remote Accelerator Calls on Hostless FPGAs](<https://devfeed.tech/articles/frac-remote-accelerator-calls-on-hostless-fpgas-127829.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2504.04404>)

Author: Ziyi Yang, Krishnan B. Iyer, Yixi Chen, Ran Shu, Zsolt Istv\\'an, Marco Canini, Suhaib A. Fahmy

Published: 2026-11-11T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Networking](<https://devfeed.tech/sources/arxiv-networking.md>)

Topics: [Remote Procedure Call (RPC)](<https://devfeed.tech/topics/rpc.md>), [rdma](<https://devfeed.tech/topics/rdma.md>), [Serverless](<https://devfeed.tech/topics/serverless.md>)

Tags: [accelerators](<https://devfeed.tech/tags/accelerators.md>), [cpu](<https://devfeed.tech/tags/cpu.md>), [cs-ar](<https://devfeed.tech/tags/cs-ar.md>), [cs-ni](<https://devfeed.tech/tags/cs-ni.md>), [distributed](<https://devfeed.tech/tags/distributed.md>), [dpu](<https://devfeed.tech/tags/dpu.md>), [fpgas](<https://devfeed.tech/tags/fpgas.md>), [functions](<https://devfeed.tech/tags/functions.md>), [hardware](<https://devfeed.tech/tags/hardware.md>), [interface](<https://devfeed.tech/tags/interface.md>), [latency](<https://devfeed.tech/tags/latency.md>), [optimized](<https://devfeed.tech/tags/optimized.md>), [overhead](<https://devfeed.tech/tags/overhead.md>), [power](<https://devfeed.tech/tags/power.md>), [prototype](<https://devfeed.tech/tags/prototype.md>), [remote](<https://devfeed.tech/tags/remote.md>), [request](<https://devfeed.tech/tags/request.md>), [rpc](<https://devfeed.tech/tags/rpc.md>), [runtime](<https://devfeed.tech/tags/runtime.md>), [throughput](<https://devfeed.tech/tags/throughput.md>), [transport](<https://devfeed.tech/tags/transport.md>), [under](<https://devfeed.tech/tags/under.md>)

### AI overview

fRAC moves network-to-accelerator invocation into FPGA hardware: it handles transport ingestion, request reassembly, dispatch, and accelerator launch without host software. The prototype is a lightweight shim on an FPGA hosting multiple virtualized accelerators, which can be reconfigured in a few milliseconds. The authors report under 7.8 μs end-to-end latency and up to 90 Gbps application throughput. Against optimized DPDK-based CPU and DPU baselines, fRAC reports at least 80% lower latency under saturation and 85% less power. The abstract identifies serverless functions using fine-grained accelerators as a suitable use case because host invocation overhead can match or exceed their execution time.

### Source excerpt

arXiv:2504.04404v3 Announce Type: replace Abstract: Hardware accelerators are essential for meeting microsecond-scale demands in distributed applications, but invoking them over the network remains bottlenecked by host software. Host-mediated paths (RPC, RDMA) still rely on the CPU to reassemble requests and launch accelerators, which dominates end-to-end latency, while packet-level offload on SmartNICs lacks any notion of a complete request. We introduce the Remote Accelerator Call (RAC) paradigm, in which transport ingestion, request reassembly, dispatch, and accelerator invocation are handled entirely in hardware. We build fRAC, a hostless prototype that implements this network-to-accelerator interface as a lightweight shim on an FPGA hosting multiple virtualized accelerators, reconfigurable at runtime in a few milliseconds. Serverless functions backed by fine-grained accelerators are a natural fit, since host-mediated invocation overhead can match or exceed their short execution times. fRAC achieves under 7.8us end-to-end latency while sustaining application throughput of up to 90 Gbps. Compared to highly optimized DPDK-based CPU and DPU baselines, it achieves at least 80% lower latency under saturation, while consuming 85% less power.

## Placing LLM Lookup Tables by Per-Layer Prefetch Deadlines

DevFeed: [Placing LLM Lookup Tables by Per-Layer Prefetch Deadlines](<https://devfeed.tech/articles/placing-llm-lookup-tables-by-per-layer-prefetch-deadlines-127806.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.13413>)

Author: Wen Huang, Jack Liu

Published: 2026-11-11T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Distributed Computing](<https://devfeed.tech/sources/arxiv-distributed-computing.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Cache](<https://devfeed.tech/topics/cache.md>), [NVMe](<https://devfeed.tech/topics/nvme.md>), [rdma](<https://devfeed.tech/topics/rdma.md>), [IO](<https://devfeed.tech/topics/io.md>), [jetstream](<https://devfeed.tech/topics/jetstream.md>)

Tags: [1-4](<https://devfeed.tech/tags/1-4.md>), [agent](<https://devfeed.tech/tags/agent.md>), [cache](<https://devfeed.tech/tags/cache.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-dc](<https://devfeed.tech/tags/cs-dc.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cuda](<https://devfeed.tech/tags/cuda.md>), [decode](<https://devfeed.tech/tags/decode.md>), [deepseek](<https://devfeed.tech/tags/deepseek.md>), [device](<https://devfeed.tech/tags/device.md>), [dram](<https://devfeed.tech/tags/dram.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [graph](<https://devfeed.tech/tags/graph.md>), [host-memory](<https://devfeed.tech/tags/host-memory.md>), [kv-cache](<https://devfeed.tech/tags/kv-cache.md>), [layer](<https://devfeed.tech/tags/layer.md>), [layers](<https://devfeed.tech/tags/layers.md>), [llm](<https://devfeed.tech/tags/llm.md>), [lookup](<https://devfeed.tech/tags/lookup.md>), [median](<https://devfeed.tech/tags/median.md>), [memory](<https://devfeed.tech/tags/memory.md>), [model](<https://devfeed.tech/tags/model.md>), [tables](<https://devfeed.tech/tags/tables.md>)

### AI overview

SlackTier places Engram lookup tables according to each layer's prefetch deadline, session reuse, and shared-device contention. In SGLang it starts lookups at step start, prioritizes the short-window layer, and divides a small GPU row cache across layers; the decode CUDA-graph path is bit-exact with the original gather. Layer 14 can use uncached NVMe without measurable decode cost. Putting both tables on four NVMe disks raises mean time per output token by at most 1.4% on agent and chat replays while reducing host memory from 212 to 34 GiB. With RDMA from another node's memory, both tables need no cache and one copy serves two instances. In a KV-constrained agent workload, using the freed memory for host KV cache at the same budget increased median throughput 36% and reduced median p90 time to first token from 38 to 24 seconds. The abstract notes that static hot sets transfer poorly for agent and chat traffic, which reuses rows within sessions.

### Source excerpt

arXiv:2611.13413v1 Announce Type: new Abstract: DeepSeek-V4.1's 203 GB of Engram n-gram tables sit in host DRAM today, taking 190 of the 223 GiB a 4-GPU serving process uses. Row indices depend only on input tokens, so reads can start with each step and overlap earlier layers. V4.1's two Engram layers have windows that differ by 30x (0.3 vs. 9.3 ms). We show that this deadline, together with session reuse and contention on a shared device, decides which tier each table can use and which layers need cache, and we give a model that predicts the resulting slowdown from measured read times. Agent and chat traffic reuse rows within sessions, and static hot sets transfer poorly. We build SlackTier, a table tier for SGLang that issues all lookups at step start. It reads the short-window layer first and splits a small GPU row cache across layers. The path runs inside the decode CUDA graph and is bit-exact with the original gather. Layer 14's table can go to NVMe uncached with no measurable decode cost. With both tables on four NVMe disks, mean time per output token on agent and chat replays rises at most 1.4% over unmodified SGLang, and host memory falls from 212 to 34 GiB. Both tables need no cache when read over RDMA (remote direct memory access) from another node's memory; one copy serves two instances. On a KV-constrained agent workload, using the freed memory as host KV cache at the same memory budget raises median throughput by 36% and cuts median p90 time to first token from 38 to 24 s.

## Release Sooner, Expose Less: Offloading MoE Dispatch in NCCL-EP under Dual-Batch Overlap

DevFeed: [Release Sooner, Expose Less: Offloading MoE Dispatch in NCCL-EP under Dual-Batch Overlap](<https://devfeed.tech/articles/release-sooner-expose-less-offloading-moe-dispatch-in-nccl-ep-under-dual-batch-overlap-126744.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.12362>)

Author: Hsin-Chun Yin, Khaled Hamidouche, Ian Ziemba, Arjun Kashyap, Pak Markthub, Subhadeep Bhattacharya, Ke Wen, Assaf Ravid, Manjunath Gorentla Venkata, Gil Bloch

Published: 2026-11-10T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Distributed Computing](<https://devfeed.tech/sources/arxiv-distributed-computing.md>)

Topics: [Mixture of Experts (MoE)](<https://devfeed.tech/topics/mixture-of-experts-moe.md>), [gpudirect rdma](<https://devfeed.tech/topics/gpudirect-rdma.md>), [rdma](<https://devfeed.tech/topics/rdma.md>), [vllm](<https://devfeed.tech/topics/vllm.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [accessible](<https://devfeed.tech/tags/accessible.md>), [after](<https://devfeed.tech/tags/after.md>), [all](<https://devfeed.tech/tags/all.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [batch](<https://devfeed.tech/tags/batch.md>), [but](<https://devfeed.tech/tags/but.md>), [can](<https://devfeed.tech/tags/can.md>), [communication](<https://devfeed.tech/tags/communication.md>), [configuration](<https://devfeed.tech/tags/configuration.md>), [cs-dc](<https://devfeed.tech/tags/cs-dc.md>), [deepep](<https://devfeed.tech/tags/deepep.md>), [deferred](<https://devfeed.tech/tags/deferred.md>), [dispatch](<https://devfeed.tech/tags/dispatch.md>), [less](<https://devfeed.tech/tags/less.md>), [moe](<https://devfeed.tech/tags/moe.md>), [nccl](<https://devfeed.tech/tags/nccl.md>), [offloading](<https://devfeed.tech/tags/offloading.md>), [overlap](<https://devfeed.tech/tags/overlap.md>), [release](<https://devfeed.tech/tags/release.md>), [under](<https://devfeed.tech/tags/under.md>)

### AI overview

Dual-batch overlap hides MoE network progress only after dispatch send releases the GPU, so send time remains exposed. On a GB300 NVL72, three NCCL-EP configurations reduce dispatch-send time versus default GDAKI across most evaluated operating points: GPI has the GPU build GPU-Friendly Descriptors (GFDs) for NIC-accessible transport; GPI-MC represents token fan-out in one GFD; and GDAKI Free Flow advances sends on a host thread. Offload lengthens the in-flight interval, and 32-512 microseconds of independent work between send and receive can hide deferred progress, leaving less exposed time than default GDAKI, depending on configuration. In disaggregated vLLM serving with DBO, median TPOT falls 15-25% versus default GDAKI and 16-42% versus DeepEP v1. These are results for the reported system and evaluated sweep.

### Source excerpt

arXiv:2611.12362v1 Announce Type: new Abstract: Dual-batch overlap (DBO) hides mixture-of-experts (MoE) communication behind another microbatch's computation, but only after dispatch send releases the GPU. The time dispatch send holds the GPU therefore stays exposed, while network progress after release can be hidden. We evaluate three research configurations built on NCCL-EP that target this exposed communication time: GPI, in which the GPU constructs GPU-Friendly Descriptors (GFDs) and submits them to NIC-accessible memory for transport; GPI-MC, which describes MoE token fan-out in one GFD; and GDAKI Free Flow, which moves send progression to a host thread. On a GB300 NVL72 system, all three approaches reduce dispatch-send time relative to default GDAKI over most operating points in the evaluated parameter sweep. An experiment that inserts a GPU delay between dispatch send and receive shows that offload increases the in-flight interval. Depending on the configuration, 32 to 512 microseconds of computation or other independent work between send and receive hides the deferred progress, leaving less exposed time than default GDAKI. In disaggregated vLLM serving with DBO, these approaches lower median TPOT by 15-25% relative to NCCL-EP default GDAKI and by 16-42% relative to DeepEP v1.

## MTRDMA: Lightweight User-Space Control-Plane Isolation for Multi-Tenant RDMA

DevFeed: [MTRDMA: Lightweight User-Space Control-Plane Isolation for Multi-Tenant RDMA](<https://devfeed.tech/articles/mtrdma-lightweight-user-space-control-plane-isolation-for-multi-tenant-rdma-126800.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.11991>)

Author: Hongyu Wang, Peihao Huang, Bangbo Liang, Lei Wang, Guo Chen

Published: 2026-11-10T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Networking](<https://devfeed.tech/sources/arxiv-networking.md>)

Topics: [control-plane](<https://devfeed.tech/topics/control-plane.md>), [rdma](<https://devfeed.tech/topics/rdma.md>), [cgroups](<https://devfeed.tech/topics/cgroups.md>), [Process](<https://devfeed.tech/topics/process.md>), [thread safe data sharing](<https://devfeed.tech/topics/thread-safe-data-sharing.md>)

Tags: [atomic](<https://devfeed.tech/tags/atomic.md>), [cache](<https://devfeed.tech/tags/cache.md>), [cgroup](<https://devfeed.tech/tags/cgroup.md>), [collector](<https://devfeed.tech/tags/collector.md>), [concurrent](<https://devfeed.tech/tags/concurrent.md>), [containers](<https://devfeed.tech/tags/containers.md>), [control-plane](<https://devfeed.tech/tags/control-plane.md>), [cs-ni](<https://devfeed.tech/tags/cs-ni.md>), [isolation](<https://devfeed.tech/tags/isolation.md>), [ledger](<https://devfeed.tech/tags/ledger.md>), [lightweight](<https://devfeed.tech/tags/lightweight.md>), [networking](<https://devfeed.tech/tags/networking.md>), [overhead](<https://devfeed.tech/tags/overhead.md>), [pair](<https://devfeed.tech/tags/pair.md>), [quota](<https://devfeed.tech/tags/quota.md>), [rdma](<https://devfeed.tech/tags/rdma.md>), [region](<https://devfeed.tech/tags/region.md>), [user](<https://devfeed.tech/tags/user.md>), [zero-copy](<https://devfeed.tech/tags/zero-copy.md>)

### AI overview

MTRDMA isolates RDMA control-plane resources on commodity RNICs to prevent one tenant's activity from degrading others. It intercepts library-level control-plane operations and enforces hierarchical quotas while preserving the zero-copy data path. Its mechanisms include atomic shared-memory quota state with per-process cached admission (~0.13 μs), a hybrid collector that reclaims orphaned quota within seconds, and rdma-cgroup caps with ledger-kernel reconciliation to bound and detect residual tampering. On ConnectX-6, it reduced victim degradation under Queue Pair cache exhaustion from 94.5% to 0.08%, also protected against Memory Region cache exhaustion, and supported 256 concurrent tenants with negligible data-plane overhead.

### Source excerpt

arXiv:2611.11991v1 Announce Type: new Abstract: In managed multi-tenant clouds, RDMA's lack of control-plane (CP) resource isolation lets a noisy tenant exhaust shared NIC microarchitectural caches, degrading victim throughput by up to 96.4\%. Data-path schedulers leave CP resources unprotected, and hardware approaches require FPGAs or SmartNICs rather than the commodity RNICs operators already deploy. We present MTRDMA, a lightweight user-space CP isolation mechanism for commodity RNICs, applicable to managed runtimes such as containers and sandboxed processes. MTRDMA intercepts library-level CP operations to enforce hierarchical quotas while preserving the zero-copy data path, using atomic shared-memory quota state with per-process cached admission (${\sim}$0.13\,$\mu$s), a hybrid garbage collector that reclaims orphaned quota within seconds, and rdma-cgroup caps with ledger--kernel reconciliation to bound and detect residual tampering. On ConnectX-6 RNICs, MTRDMA cuts victim degradation under Queue Pair (QP) cache exhaustion from 94.5\% to 0.08\%, extends protection to Memory Region (MR) cache exhaustion, and scales to 256 concurrent tenants with negligible data-plane overhead.

## Performance Bottlenecks in Computing ZK Proofs on a GPU

DevFeed: [Performance Bottlenecks in Computing ZK Proofs on a GPU](<https://devfeed.tech/articles/performance-bottlenecks-in-computing-zk-proofs-on-a-gpu-122805.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.09569>)

Author: Victor Carolino, Caleb Geren, Michael Kaufman, Maximilian Nussbaum, William Scott, Henry F. Korth

Published: 2026-11-09T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Cryptography and Security](<https://devfeed.tech/sources/arxiv-cryptography-and-security.md>)

Topics: [Zero-knowledge proof](<https://devfeed.tech/topics/zkp.md>), [Multi-GPU](<https://devfeed.tech/topics/multi-gpu.md>), [GPU optimization](<https://devfeed.tech/topics/gpu-optimization.md>), [rdma](<https://devfeed.tech/topics/rdma.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [algorithms](<https://devfeed.tech/tags/algorithms.md>), [bottlenecks](<https://devfeed.tech/tags/bottlenecks.md>), [circuit](<https://devfeed.tech/tags/circuit.md>), [communication](<https://devfeed.tech/tags/communication.md>), [computing](<https://devfeed.tech/tags/computing.md>), [constraint](<https://devfeed.tech/tags/constraint.md>), [cost](<https://devfeed.tech/tags/cost.md>), [cs-cr](<https://devfeed.tech/tags/cs-cr.md>), [data](<https://devfeed.tech/tags/data.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [gpus](<https://devfeed.tech/tags/gpus.md>), [hardware](<https://devfeed.tech/tags/hardware.md>), [low-level](<https://devfeed.tech/tags/low-level.md>), [memory](<https://devfeed.tech/tags/memory.md>), [memory-management](<https://devfeed.tech/tags/memory-management.md>), [multi-gpu](<https://devfeed.tech/tags/multi-gpu.md>), [performance](<https://devfeed.tech/tags/performance.md>), [proofs](<https://devfeed.tech/tags/proofs.md>), [zk-proofs](<https://devfeed.tech/tags/zk-proofs.md>)

### AI overview

The paper examines GPU acceleration of PlonK proof generation for applications that do not suit incrementally verifiable computing, focusing on how greater parallelism changes performance while increasing memory-management demands. It implements a parallelized PlonK to study these trade-offs and the proving protocol's data flows. For large circuits, it considers multi-GPU setups, which introduce communication, hardware-management, partitioning, and transfer challenges. The abstract also discusses motivating use cases, cost models, and RDMA for transfers between GPUs; it provides no specific benchmark results or implementation recommendations.

### Source excerpt

arXiv:2611.09569v1 Announce Type: new Abstract: Zero-knowledge proofs allow one party to prove the correctness of a statement or execution of a computation while revealing no more than the statement of correctness itself. Recent use cases, such as AI datasets, suggest the need for faster proof generation over increasingly large constraint sets. This paper explores efforts in optimizing zero-knowledge proof generation, specifically PlonK, using GPUs, for applications that are not amenable to solution via incrementally verifiable computing (IVC). Our intention with this work is to improve ZK performance by introducing fundamental changes in the low-level architectural structure of proof generation. These changes affect performance curves for most current proof generation algorithms. In particular, this paper explores the trade-offs between increased parallelization on GPUs and the resultant memory management problems that occur due to this parallelization. We implement a parallelized PlonK in order to analyze these trade-offs and make arguments concerning the nature of data flows within the proving protocol. Our results lead us to consider multi-GPU environments to maximize performance for large circuit sizes. Such environments introduce issues of communication, efficient hardware management, data partition, and data transfers. This paper further discusses the use cases that motivate multi-GPU implementations, their cost models, and the use of remote direct memory access (RDMA) for data transfer among GPUs.

## Yunsilicon Working To Upstream Their High Performance Linux Network Driver

DevFeed: [Yunsilicon Working To Upstream Their High Performance Linux Network Driver](<https://devfeed.tech/articles/yunsilicon-working-to-upstream-their-high-performance-linux-network-driver-122508.md>)

Original publisher: [Read original article](<https://www.phoronix.com/news/Yunsilicon-Linux-Network-Driver>)

Author: Michael Larabel

Published: 2026-11-08T11:27:33Z

Content type: news

Language: en

Sources: [Phoronix](<https://devfeed.tech/sources/phoronix.md>)

Topics: [Linux Kernel](<https://devfeed.tech/topics/linux-kernel.md>), [Linux](<https://devfeed.tech/topics/linux.md>), [rdma](<https://devfeed.tech/topics/rdma.md>)

Tags: [desktop-linux](<https://devfeed.tech/tags/desktop-linux.md>), [driver](<https://devfeed.tech/tags/driver.md>), [hardware](<https://devfeed.tech/tags/hardware.md>), [kernel](<https://devfeed.tech/tags/kernel.md>), [linux](<https://devfeed.tech/tags/linux.md>), [linux-benchmarking](<https://devfeed.tech/tags/linux-benchmarking.md>), [linux-hardware-benchmarks](<https://devfeed.tech/tags/linux-hardware-benchmarks.md>), [linux-hardware-reviews](<https://devfeed.tech/tags/linux-hardware-reviews.md>), [linux-how-to](<https://devfeed.tech/tags/linux-how-to.md>), [linux-kernel](<https://devfeed.tech/tags/linux-kernel.md>), [linux-performance](<https://devfeed.tech/tags/linux-performance.md>), [linux-server-benchmarks](<https://devfeed.tech/tags/linux-server-benchmarks.md>), [mainline](<https://devfeed.tech/tags/mainline.md>), [network](<https://devfeed.tech/tags/network.md>), [networking](<https://devfeed.tech/tags/networking.md>), [open-source-graphics](<https://devfeed.tech/tags/open-source-graphics.md>), [phoronix](<https://devfeed.tech/tags/phoronix.md>), [phoronix-test-suite](<https://devfeed.tech/tags/phoronix-test-suite.md>), [ubuntu-benchmarks](<https://devfeed.tech/tags/ubuntu-benchmarks.md>), [ubuntu-hardware](<https://devfeed.tech/tags/ubuntu-hardware.md>), [upstream](<https://devfeed.tech/tags/upstream.md>)

### AI overview

Yunsilicon submitted a three-driver series seeking mainline Linux support for a dual-function PCIe adapter (vendor 0x1f67) with Ethernet and RDMA. The shared xsc_pci core owns the PCI function and common queues, interrupts, registers, and resource pools, then publishes an auxiliary device for each function. The xsc_eth and xsc_ib drivers bind to that device, sharing the core's probe and resources. Ethernet support includes TX/RX paths using NAPI and page_pool, RSS, checksum offloads, and statistics; RDMA support registers an ib_device and provides verbs and data paths, with the netdev supplying port state and GID. The excerpt says the GPLv2 driver has been in development since 2021 and that patches appeared on LKML the previous month, followed by a more complete submission this week. It does not establish whether the series has been accepted into mainline.

### Source excerpt

Yunsilicon is a Chinese semiconductor startup founded in 2021 that is working on producing high performance networking adapters and DPUs. Over the past month has been their first foray into posting patches to the Linux kernel mailing list and working to get their networking hardware supported by the mainline Linux kernel...

## Scheduling Data to Compute: Hierarchical Data Pipelining for LLM Training on AMD GPUs

DevFeed: [Scheduling Data to Compute: Hierarchical Data Pipelining for LLM Training on AMD GPUs](<https://devfeed.tech/articles/scheduling-data-to-compute-hierarchical-data-pipelining-for-llm-training-on-amd-gpus-118373.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.05751>)

Author: Gengyu Rao, Ghadeer Ahmed H Alabandi, Su Ann Chong, Thomas Huber, Kailash Gogineni, Subhajit Dutta Chowdhury, Saleel Kudchadker, Suchita Pati, Zirui Zhang, Zhen Huang, Yanyuan Qin, Xiaoming Peng, Lihu

Published: 2026-11-05T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>), [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [distributed-training](<https://devfeed.tech/topics/distributed-training.md>), [rdma](<https://devfeed.tech/topics/rdma.md>), [dma](<https://devfeed.tech/topics/dma.md>)

Tags: [amd](<https://devfeed.tech/tags/amd.md>), [bandwidth](<https://devfeed.tech/tags/bandwidth.md>), [communication](<https://devfeed.tech/tags/communication.md>), [compute](<https://devfeed.tech/tags/compute.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-ar](<https://devfeed.tech/tags/cs-ar.md>), [data](<https://devfeed.tech/tags/data.md>), [data-movement](<https://devfeed.tech/tags/data-movement.md>), [dma](<https://devfeed.tech/tags/dma.md>), [features](<https://devfeed.tech/tags/features.md>), [first](<https://devfeed.tech/tags/first.md>), [framework](<https://devfeed.tech/tags/framework.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [gpus](<https://devfeed.tech/tags/gpus.md>), [hardware](<https://devfeed.tech/tags/hardware.md>), [hbm](<https://devfeed.tech/tags/hbm.md>), [high-bandwidth-memory](<https://devfeed.tech/tags/high-bandwidth-memory.md>), [llm](<https://devfeed.tech/tags/llm.md>), [llm-training](<https://devfeed.tech/tags/llm-training.md>), [pipelining](<https://devfeed.tech/tags/pipelining.md>), [scheduling](<https://devfeed.tech/tags/scheduling.md>), [training](<https://devfeed.tech/tags/training.md>)

### AI overview

The paper targets GPU compute stalls caused by tensors arriving late from memory or other GPUs. It pipelines explicit data movement to keep compute units supplied: Tensor Data Mover (TDM) transfers tensors from HBM to on-chip memory, transpose-load instructions arrange data in the needed layout during loading, and SDMA/RDMA move data between GPUs over scale-up/scale-out networks. The abstract reports average kernel speedups of 1.88x for TDM and 1.19x for transpose-load, plus 1.16x and 1.33x for SDMA and RDMA when computation overlaps communication. The implementations are integrated into open-source AMD GPU training software.

### Source excerpt

arXiv:2611.05751v1 Announce Type: cross Abstract: Large language models (LLM) have been the key focus of recent machine learning breakthroughs. As the model size grows exponentially, the computation needed for training is astronomical. A GPU can do a huge amount of arithmetic. However, the data its compute units need is not always available when they are ready. This data may still be in slow memory or on another GPU. While the chip waits, expensive compute sits idle, and at scale even a small drop in utilization translates into substantial cost. In this paper, we treat this waiting time as the primary bottleneck. We focus on hardware features designed for explicit data movement. These features load data from off-chip memory onto the chip, rearrange it into the layout the computation requires as it is loaded, and transfer it between GPUs. By orchestrating them, we keep a steady pipeline of data flowing to the compute units so that they remain busy. We present kernel and framework implementations that address this bottleneck using three key data-movement instructions and hardware features. The Tensor Data Mover (TDM) moves tensor data from high-bandwidth memory (HBM) into on-chip memory. Transpose-load instructions transpose tensor data into the layout that LLM computations require as it is loaded on-chip. Finally, System DMA (SDMA) and Remote DMA (RDMA) move data between GPUs over the scale-up and scale-out networks, respectively. Our results show that TDM and transpose-load provide on average 1.88x and 1.19x speedup at the kernel level, respectively. The SDMA and RDMA lead to on average 1.16x and 1.33x speedup for computation overlapped with communication. This is the first AMD LLM training system to utilize all the GPU data pipelining features. Our implementation has been integrated into open source framework and library for AMD GPUs. https://github.com/AMD-AGI/Primus-Turbo https://github.com/AMD-AGI/Primus

## The Endgame of FSDP Communication: Layout-Aware Collectives on Copy Engines, RDMA, and NVSwitch

DevFeed: [The Endgame of FSDP Communication: Layout-Aware Collectives on Copy Engines, RDMA, and NVSwitch](<https://devfeed.tech/articles/the-endgame-of-fsdp-communication-layout-aware-collectives-on-copy-engines-rdma-and-nvswitch-107569.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.38228>)

Author: Zejian Wang, Yifan Zhao, Kaihui Ren, Xiaoling Zhou, Yubin Wang, Chenhao Guan, Lin Zhang, Yufeng Li, Zhikang Mo, Zhuoru Ma, Yingtong Xiong, Jihu Guo, Yiqiao Liao, Chengru Song

Published: 2026-10-30T04:00:00Z

Content type: article

Language: en

Sources: [arXiv: Distributed Computing](<https://devfeed.tech/sources/arxiv-distributed-computing.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [rdma](<https://devfeed.tech/topics/rdma.md>), [Distributed Systems & Parallel Computing](<https://devfeed.tech/topics/distributed-systems-parallel-computing.md>), [NCCL](<https://devfeed.tech/topics/nccl.md>)

Tags: [collective](<https://devfeed.tech/tags/collective.md>), [communication](<https://devfeed.tech/tags/communication.md>), [compute](<https://devfeed.tech/tags/compute.md>), [concurrent](<https://devfeed.tech/tags/concurrent.md>), [conversion](<https://devfeed.tech/tags/conversion.md>), [copy](<https://devfeed.tech/tags/copy.md>), [cross](<https://devfeed.tech/tags/cross.md>), [cs-dc](<https://devfeed.tech/tags/cs-dc.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-pf](<https://devfeed.tech/tags/cs-pf.md>), [data](<https://devfeed.tech/tags/data.md>), [engines](<https://devfeed.tech/tags/engines.md>), [gradient](<https://devfeed.tech/tags/gradient.md>), [kernels](<https://devfeed.tech/tags/kernels.md>), [layout](<https://devfeed.tech/tags/layout.md>), [layouts](<https://devfeed.tech/tags/layouts.md>), [llama](<https://devfeed.tech/tags/llama.md>), [magnitude](<https://devfeed.tech/tags/magnitude.md>), [memory](<https://devfeed.tech/tags/memory.md>), [multicast](<https://devfeed.tech/tags/multicast.md>), [nccl](<https://devfeed.tech/tags/nccl.md>), [nvswitch](<https://devfeed.tech/tags/nvswitch.md>), [rdma](<https://devfeed.tech/tags/rdma.md>)

### AI overview

TIDE targets two FSDP2 costs: copying between per-parameter tensors and flat collective buffers, and communication kernels competing with compute. It folds placement into collective addressing while preserving FSDP2 storage and compute kernels. TIDE-AG writes shards directly to parameter storage using NVSwitch multicast and paced recursive-doubling RDMA without using SMs for transport or placement. TIDE-RS reads per-parameter bf16 gradients and produces fp32 shards through copy engines, short-lived CTAs for node-local accumulation, and NVSwitch for cross-node reduction; its two-phase pipeline separates gradient reads from network progress, with optional Slot-Reuse. In reported 64-GPU experiments across Llama-3.1 70B, Wan2.2, and Qwen3-30B-A3B, TIDE cut step time by up to 27.1% versus Stock FSDP2 and 6.0% versus FSDP2+SymmMem. The abstract also reports 12.5-14.5x lower reduce-scatter SM time and up to 4.7 GiB less peak memory than FSDP2+SymmMem. These are reported maxima; the supplied evidence is an abstract, not full paper text.

### Source excerpt

arXiv:2610.38228v1 Announce Type: new Abstract: Fully Sharded Data Parallelism (FSDP2) maintains parameters and gradients as per-parameter tensors, whereas its collective path operates on flat, rank-major buffers. Converting between these layouts copies all-gather outputs into parameter storage and packs gradients into full-group fp32 buffers. Layout conversion alone accounts for 6.4% of step time in Llama-3.1 70B training at 8K tokens per GPU. Communication kernels impose a second cost: a default NCCL reduce-scatter can slow a concurrent GEMM that would otherwise complete in a single wave by 69%. We present TIDE, a set of zero-copy collectives built on public NCCL 2.31+ APIs. TIDE preserves FSDP2's per-parameter storage and compute kernels while folding layout placement into collective addressing. TIDE-AG delivers shards directly to parameter storage using NVSwitch multicast and paced recursive-doubling RDMA, with no SMs used for transport or placement. TIDE-RS consumes per-parameter bf16 gradients directly and reduces them into fp32 shards (a mixed-precision mode that NCCL does not provide) using copy engines and short-lived thread blocks (CTAs) for node-local accumulation and NVSwitch for cross-node reduction. A two-phase pipeline separates gradient reads from network progress; Slot-Reuse optionally stages gradients in vacated all-gather slots. On 64 GPUs, TIDE reduces step time by up to 27.1% relative to Stock FSDP2 and 6.0% relative to FSDP2+SymmMem across Llama-3.1 70B, Wan2.2, and Qwen3-30B-A3B. It reduces reduce-scatter SM time by factors of 12.5-14.5 and cuts interference with concurrent compute by up to an order of magnitude. Its reduce-scatter buffer remains nearly constant from 16 to 64 GPUs at a fixed per-GPU output size, and its peak memory is up to 4.7 GiB below FSDP2+SymmMem's and, with Slot-Reuse on Wan2.2, below Stock FSDP2's.

## Oracle Acceleron Adds Front-End RDMA, an NCCL and RCCL Plugin for Multiplanar Failover, and 3x NVMe Block Storage Performance on OCI

DevFeed: [Oracle Acceleron Adds Front-End RDMA, an NCCL and RCCL Plugin for Multiplanar Failover, and 3x NVMe Block Storage Performance on OCI](<https://devfeed.tech/articles/oracle-acceleron-adds-front-end-rdma-an-nccl-and-rccl-plugin-for-multiplanar-failover-and-3x-nvme-block-storage-performance-on-oci-105442.md>)

Original publisher: [Read original article](<https://www.storagereview.com/news/oracle-acceleron-front-end-rdma-occl-plugin-multiplanar-failover-3x-nvme-block-storage>)

Author: Harold Fritts

Published: 2026-10-28T14:55:26Z

Content type: news

Language: en

Sources: [StorageReview.com](<https://devfeed.tech/sources/storagereview-com.md>)

Topics: [rdma](<https://devfeed.tech/topics/rdma.md>), [NVMe](<https://devfeed.tech/topics/nvme.md>), [Load Balancing](<https://devfeed.tech/topics/load-balancing.md>), [systems](<https://devfeed.tech/topics/systems.md>), [AI Networking](<https://devfeed.tech/topics/ai-networking.md>)

Tags: [acceleron](<https://devfeed.tech/tags/acceleron.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-world](<https://devfeed.tech/tags/ai-world.md>), [bare-metal](<https://devfeed.tech/tags/bare-metal.md>), [block](<https://devfeed.tech/tags/block.md>), [capabilities](<https://devfeed.tech/tags/capabilities.md>), [cloud](<https://devfeed.tech/tags/cloud.md>), [cloud-infrastructure](<https://devfeed.tech/tags/cloud-infrastructure.md>), [enterprise](<https://devfeed.tech/tags/enterprise.md>), [failover](<https://devfeed.tech/tags/failover.md>), [front-end](<https://devfeed.tech/tags/front-end.md>), [infrastructure](<https://devfeed.tech/tags/infrastructure.md>), [nccl](<https://devfeed.tech/tags/nccl.md>), [network](<https://devfeed.tech/tags/network.md>), [networking](<https://devfeed.tech/tags/networking.md>), [nvme](<https://devfeed.tech/tags/nvme.md>), [oci](<https://devfeed.tech/tags/oci.md>), [oracle](<https://devfeed.tech/tags/oracle.md>), [performance](<https://devfeed.tech/tags/performance.md>), [plugin](<https://devfeed.tech/tags/plugin.md>), [rdma](<https://devfeed.tech/tags/rdma.md>), [storage](<https://devfeed.tech/tags/storage.md>)

### AI overview

Oracle's new Oracle Acceleron capabilities add RDMA to OCI's shared front-end network, NVMe block storage attachments for new bare metal and VM shapes, and multiplanar failover for front-end traffic. Oracle says existing RDMA applications work without changes and rates the storage improvement at up to 3x, but gives no benchmark baseline, workload, or metric; availability dates are also absent. To share the front end across RDMA and TCP, the SmartNIC uses hierarchical bandwidth buckets, congestion marking or drops, and delay-based path probing. The Oracle Collective Communications Library (OCCL) plugin supports NVIDIA NCCL and AMD RCCL interfaces, distributes collective traffic across reliable connections, and shifts traffic away from congested or failed paths without restarting a job. Oracle says OCCL works with single- or multi-plane networks and does not depend on NIC firmware or topology. The release also describes OCI Global CDN Connect partnerships and a non-binding F5 agreement for a planned security service.

### Source excerpt

Oracle announced a second round of Oracle Acceleron capabilities at Oracle AI World in Las Vegas, a year after the networking suite first bundled dedicated fabrics, multiplanar networking, SmartNICs, and direct data paths for Oracle Cloud Infrastructure. The new pieces are RDMA on OCI's shared front-end network, NVMe block storage attachments across Acceleron bare metal The post Oracle Acceleron Adds Front-End RDMA, an NCCL and RCCL Plugin for Multiplanar Failover, and 3x NVMe Block Storage Performance on OCI appeared first on StorageReview.com.

## How DRA can improve the performance of distributed LLM inference workloads with RDMA and SR-IOV

DevFeed: [How DRA can improve the performance of distributed LLM inference workloads with RDMA and SR-IOV](<https://devfeed.tech/articles/how-dra-can-improve-the-performance-of-distributed-llm-inference-workloads-with-rdma-and-sr-iov-105451.md>)

Original publisher: [Read original article](<https://developers.redhat.com/articles/2026/10/28/how-dra-can-improve-performance-of-distributed-llm-inference-workloads-with-rdma-and-sr-iov>)

Author: David Whyte-Gray, William Zhao, Raj Joshi, Sebastian Scheinkman

Published: 2026-10-28T13:16:22Z

Content type: article

Language: en

Sources: [Red Hat](<https://devfeed.tech/sources/red-hat.md>), [Red Hat Developer](<https://devfeed.tech/sources/red-hat-developer.md>)

Topics: [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [gpudirect rdma](<https://devfeed.tech/topics/gpudirect-rdma.md>), [rdma](<https://devfeed.tech/topics/rdma.md>), [performance-engineering](<https://devfeed.tech/topics/performance-engineering.md>), [vllm](<https://devfeed.tech/topics/vllm.md>), [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>), [High-Performance Computing](<https://devfeed.tech/topics/high-performance-computing.md>)

Tags: [ai-workloads](<https://devfeed.tech/tags/ai-workloads.md>), [alignment](<https://devfeed.tech/tags/alignment.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [bandwidth](<https://devfeed.tech/tags/bandwidth.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [distributed](<https://devfeed.tech/tags/distributed.md>), [dra](<https://devfeed.tech/tags/dra.md>), [llm-inference](<https://devfeed.tech/tags/llm-inference.md>), [performance](<https://devfeed.tech/tags/performance.md>), [rdma](<https://devfeed.tech/tags/rdma.md>), [vllm](<https://devfeed.tech/tags/vllm.md>)

### AI overview

GPU and RDMA NIC placement can sharply affect distributed inference performance, and Kubernetes DRA can require the two devices to share a PCIe root. Traditional device plugins allocate GPU and SR-IOV VF independently, without expressing that topology relationship. The described setup uses the NVIDIA GPU DRA driver and `dra-driver-sriov`; both publish `resource.kubernetes.io/pcieRoot`, and a ResourceClaim applies `matchAttribute` so a claim remains pending if no aligned pair is available. SR-IOV policies must select and advertise VFs, and the cluster needs DRA enabled and the relevant drivers and operator installed. In tests on an A100 platform, aligned placement delivered 194.6 Gbit/s RDMA bandwidth versus 127.1 Gbit/s for same-NUMA but different-root placement and 110.0 Gbit/s across NUMA nodes. For the tested llm-d Qwen prefill/decode workload, alignment raised KV-transfer throughput from 9,612.6 to 20,709.6 MB/s and cut mean latency from 122.32 to 56.78 ms. Results depend on the tested model, GPU, NIC, and server topology.

### Source excerpt

Modern LLM inference is rapidly shifting from a pure compute challenge to a distributed network bottleneck. To maintain high throughput and handle the massive context windows demanded by agentic workloads, production engines like vLLM and distributed LLM inference frameworks like llm-d are moving toward disaggregated, highly parallelized architectures. The post How DRA can improve the performance of distributed LLM inference workloads with RDMA and SR-IOV appeared first on Red Hat Developer.

## Meta Networking at the 2026 OCP Global Summit: Open Innovation for Scale-Up, Scale-Out, and Scale-Across AI Networks

DevFeed: [Meta Networking at the 2026 OCP Global Summit: Open Innovation for Scale-Up, Scale-Out, and Scale-Across AI Networks](<https://devfeed.tech/articles/meta-networking-at-the-2026-ocp-global-summit-open-innovation-for-scale-up-scale-out-and-scale-across-ai-networks-100675.md>)

Original publisher: [Read original article](<https://engineering.fb.com/2026/10/26/open-source/ocp-summit-2026-meta-networking-talks/>)

Author: Ravindra Sunkad; Omar Baldonado

Published: 2026-10-26T16:00:07Z

Content type: news

Language: en

Sources: [Engineering at Meta](<https://devfeed.tech/sources/engineering-at-meta.md>), [Meta ML Applications](<https://devfeed.tech/sources/meta-ml-applications.md>)

Topics: [AI Networking](<https://devfeed.tech/topics/ai-networking.md>), [rdma](<https://devfeed.tech/topics/rdma.md>), [Open Source](<https://devfeed.tech/topics/open-source.md>), [NVIDIA Spectrum-X Ethernet](<https://devfeed.tech/topics/nvidia-spectrum-x-ethernet.md>), [CI/CD](<https://devfeed.tech/topics/cicd.md>), [Test-driven development](<https://devfeed.tech/topics/tdd.md>)

Tags: [102-4t](<https://devfeed.tech/tags/102-4t.md>), [2026](<https://devfeed.tech/tags/2026.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-models](<https://devfeed.tech/tags/ai-models.md>), [ai-networks](<https://devfeed.tech/tags/ai-networks.md>), [broadcom](<https://devfeed.tech/tags/broadcom.md>), [cluster](<https://devfeed.tech/tags/cluster.md>), [data-center-engineering](<https://devfeed.tech/tags/data-center-engineering.md>), [data-infrastructure](<https://devfeed.tech/tags/data-infrastructure.md>), [devinfra](<https://devfeed.tech/tags/devinfra.md>), [global](<https://devfeed.tech/tags/global.md>), [innovation](<https://devfeed.tech/tags/innovation.md>), [meta](<https://devfeed.tech/tags/meta.md>), [ml-applications](<https://devfeed.tech/tags/ml-applications.md>), [networking](<https://devfeed.tech/tags/networking.md>), [networking-traffic](<https://devfeed.tech/tags/networking-traffic.md>), [networks](<https://devfeed.tech/tags/networks.md>), [ocp](<https://devfeed.tech/tags/ocp.md>), [open](<https://devfeed.tech/tags/open.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [scale](<https://devfeed.tech/tags/scale.md>), [scale-across](<https://devfeed.tech/tags/scale-across.md>), [scale-out](<https://devfeed.tech/tags/scale-out.md>), [scale-up](<https://devfeed.tech/tags/scale-up.md>), [summit](<https://devfeed.tech/tags/summit.md>), [up](<https://devfeed.tech/tags/up.md>)

### AI overview

Meta's 2026 OCP Summit overview presents an open, full-stack network strategy for AI clusters spanning scale-up, scale-out, and scale-across. Highlights include IcePack/IceCube liquid-cooled racks with 102.4T switches, and a flattened scale-out fabric that connects each GPU to eight planes and reduces a three-tier Clos to two tiers using NIC packet spraying and joint host-switch control. For cross-site capacity, BAG links buildings, campuses, and regions, alongside SRv6 backbones running on FBOSS and SAI. Scale-up work includes Tahan, an Ethernet switch for MTIA 300, and ESUN, whose 1.0 specification is released and 1.1 is in progress. Meta also describes OSPA, a multi-vendor switch-platform standard submitted to OCP, and open-sourcing MetaRoCE software, beginning with a user-space implementation; a DPDK version is planned. The excerpt lists talks on optics, interconnect reliability, deployment, and operation, but does not provide detailed results for most of them.

### Source excerpt

At the 2026 Open Compute Project (OCP) Global Summit, Meta shared the networking innovations that are directly supporting our latest AI models, including Muse Wonder and Muse Spark, and our gigawatt-scale Prometheus cluster. Highlights IcePack & IceCube: Meta's first liquid-cooled network rack with 102.4T switch blades (128x800G, Broadcom Tomahawk 6C and NVIDIA Spectrum-6 variants), running [...] Read More... The post Meta Networking at the 2026 OCP Global Summit: Open Innovation for Scale-Up, Scale-Out, and Scale-Across AI Networks appeared first on Engineering at Meta.

## CAIOS Achieves 7+ GB/s per GPU on NVIDIA Blackwell Ultra

DevFeed: [CAIOS Achieves 7+ GB/s per GPU on NVIDIA Blackwell Ultra](<https://devfeed.tech/articles/caios-achieves-7-gb-s-per-gpu-on-nvidia-blackwell-ultra-96091.md>)

Original publisher: [Read original article](<https://wf.coreweave.com/blog/caios-achieves-7-gb-s-per-gpu-on-nvidia-blackwell-ultra>)

Author: CoreWeave

Published: 2026-10-22T16:27:39Z

Content type: news

Language: en

Sources: [Coreweave Blog Posts](<https://devfeed.tech/sources/coreweave-blog-posts.md>)

Topics: [NVIDIA Blackwell](<https://devfeed.tech/topics/nvidia-blackwell.md>), [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [GPU](<https://devfeed.tech/topics/gpu.md>), [rdma](<https://devfeed.tech/topics/rdma.md>)

Tags: [accelerate](<https://devfeed.tech/tags/accelerate.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-workloads](<https://devfeed.tech/tags/ai-workloads.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [benchmarking](<https://devfeed.tech/tags/benchmarking.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [blackwell](<https://devfeed.tech/tags/blackwell.md>), [blackwell-ultra](<https://devfeed.tech/tags/blackwell-ultra.md>), [cache](<https://devfeed.tech/tags/cache.md>), [caching](<https://devfeed.tech/tags/caching.md>), [concurrency](<https://devfeed.tech/tags/concurrency.md>), [content-type](<https://devfeed.tech/tags/content-type.md>), [coreweave](<https://devfeed.tech/tags/coreweave.md>), [coreweave-ai-object-storage](<https://devfeed.tech/tags/coreweave-ai-object-storage.md>), [coreweave-kubernetes-service](<https://devfeed.tech/tags/coreweave-kubernetes-service.md>), [ethernet](<https://devfeed.tech/tags/ethernet.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [optimizations](<https://devfeed.tech/tags/optimizations.md>), [pipeline](<https://devfeed.tech/tags/pipeline.md>), [rdma](<https://devfeed.tech/tags/rdma.md>)

### AI overview

In tests on 16 NVIDIA Blackwell Ultra nodes, CAIOS averaged 7.02 GB/s per GPU using NVIDIA Quantum InfiniBand (RDMA), compared with 2.81 GB/s per GPU over Ethernet (TCP). The Warp S3 read benchmark used 10,000 objects of 50 MB each, 100 concurrent goroutines, and pipelining; throughput rose as objects were staged in CAIOS's LOTA cache. The tested Ethernet lab network capped aggregate throughput at 180 GB/s, while InfiniBand reached 449 GB/s. CoreWeave attributes the increase from its earlier 2 GB/s-per-GPU H100 result to fewer GPUs per node, about 50% higher throughput after moving node-to-node transfers to InfiniBand, and about 17% from LOTA pipeline optimizations. These are vendor-reported benchmark results; the text does not provide independent validation.

### Source excerpt

CoreWeave's CAIOS delivers 7+ GB/s per GPU on NVIDIA Blackwell Ultra with LOTA caching, RDMA, and pipeline optimizations to accelerate AI workloads.

## HAWK: Heterogeneity-Aware KV Cache Transfer for Efficient GPU-NPU Phase-Disaggregated LLM Inference

DevFeed: [HAWK: Heterogeneity-Aware KV Cache Transfer for Efficient GPU-NPU Phase-Disaggregated LLM Inference](<https://devfeed.tech/articles/hawk-heterogeneity-aware-kv-cache-transfer-for-efficient-gpu-npu-phase-disaggregated-llm-inference-95170.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.29873>)

Author: Junhyeok Park, Jungeun Yoon, Kyungmin Kim, Hunsoon Lee, Shinyoung Ahn, Kangho Kim

Published: 2026-10-22T04:00:00Z

Content type: article

Language: en

Sources: [arXiv: Distributed Computing](<https://devfeed.tech/sources/arxiv-distributed-computing.md>)

Topics: [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>), [rdma](<https://devfeed.tech/topics/rdma.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Data transmission](<https://devfeed.tech/topics/data-transmission.md>)

Tags: [ai-accelerators](<https://devfeed.tech/tags/ai-accelerators.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [buffer](<https://devfeed.tech/tags/buffer.md>), [cache](<https://devfeed.tech/tags/cache.md>), [communication](<https://devfeed.tech/tags/communication.md>), [compute-resources](<https://devfeed.tech/tags/compute-resources.md>), [cost-efficiency](<https://devfeed.tech/tags/cost-efficiency.md>), [cs-ar](<https://devfeed.tech/tags/cs-ar.md>), [cs-dc](<https://devfeed.tech/tags/cs-dc.md>), [disaggregation](<https://devfeed.tech/tags/disaggregation.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [heterogeneous-integration](<https://devfeed.tech/tags/heterogeneous-integration.md>), [high-performance](<https://devfeed.tech/tags/high-performance.md>), [inference](<https://devfeed.tech/tags/inference.md>), [kv-cache](<https://devfeed.tech/tags/kv-cache.md>), [large-language-model](<https://devfeed.tech/tags/large-language-model.md>), [layouts](<https://devfeed.tech/tags/layouts.md>), [llm](<https://devfeed.tech/tags/llm.md>), [llm-inference](<https://devfeed.tech/tags/llm-inference.md>), [memory](<https://devfeed.tech/tags/memory.md>), [npu](<https://devfeed.tech/tags/npu.md>), [power](<https://devfeed.tech/tags/power.md>)

### AI overview

HAWK enables GPU-to-NPU KV cache transfer for phase-disaggregated LLM inference, where GPUs handle compute-intensive prefill and NPUs handle more memory-bound decode. It establishes a direct RDMA path and allocates layout-aware RDMA buffers on the NPU side, so data sent from a conventional contiguous GPU buffer matches the target layout and can be consumed without additional rearrangement. On a system with commercially available GPUs and NPUs, the authors report performance within 7.88% of a GPU-only system under the same SLO constraints, with up to 1.26x higher power efficiency and 1.72x higher cost efficiency.

### Source excerpt

arXiv:2610.29873v1 Announce Type: cross Abstract: Large language model (LLM) inference exhibits distinct computational characteristics across its two phases: prefill is compute-intensive, whereas decode is more memory-bound. As a result, AI accelerators achieve varying levels of efficiency across the two phases. GPUs are well suited to prefill, but are often underutilized during decode. By contrast, NPUs can utilize their compute resources more effectively during decode. Motivated by this observation, we explore a heterogeneous architecture that integrates GPUs and NPUs, mapping prefill to GPUs and decode to NPUs to achieve high performance and efficiency. However, realizing such heterogeneous integration is challenging because LLM phase disaggregation requires KV cache transfer from prefill to decode (that is, from GPUs to NPUs), which is complicated by differences in inter-device communication mechanisms and KV cache tensor layouts. To address this, we propose HAWK, a transfer mechanism that enables rapid KV cache transfer across heterogeneous devices. HAWK establishes a remote direct memory access (RDMA) path for direct cross-device transfers while handling inter-device tensor layout discrepancies as part of the transfer process. In particular, HAWK allocates tensor-layout-aware RDMA buffers on the NPU side to match the target tensor layout. Leveraging these buffers, HAWK orchestrates data transfer from a conventional contiguous GPU RDMA buffer to the corresponding NPU buffers, allowing transferred tensors to be consumed directly without additional data rearrangement while maintaining transfer efficiency. We implement and evaluate HAWK on a system built with commercially available GPUs and NPUs, demonstrating performance within 7.88% of a GPU-only system under the same SLO constraints while achieving up to 1.26$\times$ higher power efficiency and 1.72$\times$ higher cost efficiency.

## SONiC momentum continues at OCP Global Summit

DevFeed: [SONiC momentum continues at OCP Global Summit](<https://devfeed.tech/articles/sonic-momentum-continues-at-ocp-global-summit-91360.md>)

Original publisher: [Read original article](<https://www.networkworld.com/article/4236689/sonic-momentum-continues-at-ocp-global-summit.html>)

Author: Michael Cooney

Published: 2026-10-21T01:25:34Z

Content type: news

Language: en

Sources: [Network World](<https://devfeed.tech/sources/network-world.md>)

Topics: [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [Cisco](<https://devfeed.tech/topics/cisco.md>), [NVIDIA Spectrum-X Ethernet](<https://devfeed.tech/topics/nvidia-spectrum-x-ethernet.md>), [rdma](<https://devfeed.tech/topics/rdma.md>), [evpn](<https://devfeed.tech/topics/evpn.md>), [observability](<https://devfeed.tech/topics/observability.md>)

Tags: [2026](<https://devfeed.tech/tags/2026.md>), [400g](<https://devfeed.tech/tags/400g.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-networking](<https://devfeed.tech/tags/ai-networking.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [artificial-intelligence-network-switches-networking](<https://devfeed.tech/tags/artificial-intelligence-network-switches-networking.md>), [azure](<https://devfeed.tech/tags/azure.md>), [bgp](<https://devfeed.tech/tags/bgp.md>), [cisco](<https://devfeed.tech/tags/cisco.md>), [data-center](<https://devfeed.tech/tags/data-center.md>), [edition-us](<https://devfeed.tech/tags/edition-us.md>), [ethernet](<https://devfeed.tech/tags/ethernet.md>), [google-cloud](<https://devfeed.tech/tags/google-cloud.md>), [hardware](<https://devfeed.tech/tags/hardware.md>), [hyperscale](<https://devfeed.tech/tags/hyperscale.md>), [hyperscalers](<https://devfeed.tech/tags/hyperscalers.md>), [language-en](<https://devfeed.tech/tags/language-en.md>), [linux](<https://devfeed.tech/tags/linux.md>), [microsoft](<https://devfeed.tech/tags/microsoft.md>), [network-switches](<https://devfeed.tech/tags/network-switches.md>), [networking](<https://devfeed.tech/tags/networking.md>), [nokia](<https://devfeed.tech/tags/nokia.md>), [ocp](<https://devfeed.tech/tags/ocp.md>), [sonic](<https://devfeed.tech/tags/sonic.md>), [storytype-news](<https://devfeed.tech/tags/storytype-news.md>), [summit](<https://devfeed.tech/tags/summit.md>)

### AI overview

SONiC's OCP Summit momentum centered on broader hardware support, faster route programming, and community growth for AI and cloud fabrics. Cisco says its validated SONiC runs on N9000 switches with Cloud Scale silicon and on N9100 switches with Nvidia Spectrum-X, with Silicon One support planned for N9000; Cisco presents this as extending its multi-silicon Nexus portfolio and enabling a shared support and routing-tool model alongside its 8000 Series. Microsoft and Nokia described community work to speed route programming and recovery after route changes, which can matter in large routing tables and AI fabrics. The SONiC Foundation added Lenovo, NADDOD, and NEXAI, which plan contributions in areas including automation, SAI, RDMA/RoCE, telemetry, and observability. The excerpt also describes SONiC as a Linux network OS that runs across vendors' switches and supports BGP, RDMA, QoS, and Ethernet/IP; AI use is described as concentrated largely in hyperscale environments.

### Source excerpt

Proponents of SONiC made a strong showing at last week's Open Compute Project (OCP) Global Summit, where vendors including Cisco, Microsoft, and Nokia championed SONiC's capabilities as an open Ethernet platform for AI networking. The Linux-based network operating system decouples network software from the underlying hardware and allows it to run on hundreds of switches and ASICs from multiple vendors. SONiC (which stands for Software for Open Networking in the Cloud) supports a full suite of network features, including Border Gateway Protocol (BGP), remote direct memory access (RDMA), QoS, and Ethernet/IP. SONiC's modularity, programmability, and cloud-based architecture make it a viable option for enterprises and hyperscalers to deploy as a traditional networking base. And increasingly, it's being used for AI networking, largely in hyperscale environments. The 650 Group forecasts that SONiC-based data center switching revenue will surge past $5 billion in 2026, driven by hyperscalers like Microsoft Azure and Google Cloud along with growth in community-hardened architecture for the neoclouds and enterprise. With that in mind, here's some of the SONiC-related news from OCP Global Summit: Extending Cisco-validated SONiC Cisco announced that its Cisco-validated SONiC is available on its N9000 Series with Cisco Cloud Scale silicon for high-density 100G and 400G enterprise and web-scale leaf-spine tiers, and on the Cisco N9100 Series with Nvidia Spectrum-X ASICs for high-throughput, loss-sensitive backend AI fabrics. The vendor also said it is extending its SONiC offering to the Cisco Silicon One ASIC-powered N9000 Series for ultra-high-density fabrics. Cisco already supports SONiC on its fixed and modular 8000 Series routers. This will complete SONiC integration across the Cisco multi-silicon Nexus switching portfolio, according to a blog post by Surbhi Paul, director, product management, for data center AI networking at Cisco. "For organizations deploying Cisco 8000 S

## Introducing TorchTPU

DevFeed: [Introducing TorchTPU](<https://devfeed.tech/articles/introducing-torchtpu-91073.md>)

Original publisher: [Read original article](<https://pytorch.org/blog/introducing-torchtpu/>)

Author: The TorchTPU Team†

Published: 2026-10-20T16:02:30Z

Content type: article

Language: en

Sources: [PyTorch](<https://devfeed.tech/sources/blog-pytorch.md>)

Topics: [distributed-training](<https://devfeed.tech/topics/distributed-training.md>), [Open Source](<https://devfeed.tech/topics/open-source.md>), [rdma](<https://devfeed.tech/topics/rdma.md>)

Tags: [2](<https://devfeed.tech/tags/2.md>), [2020](<https://devfeed.tech/tags/2020.md>), [2025](<https://devfeed.tech/tags/2025.md>), [2d-mesh](<https://devfeed.tech/tags/2d-mesh.md>), [3](<https://devfeed.tech/tags/3.md>), [3d](<https://devfeed.tech/tags/3d.md>), [backend](<https://devfeed.tech/tags/backend.md>), [blog](<https://devfeed.tech/tags/blog.md>), [collaboration](<https://devfeed.tech/tags/collaboration.md>), [execution](<https://devfeed.tech/tags/execution.md>), [first](<https://devfeed.tech/tags/first.md>), [google](<https://devfeed.tech/tags/google.md>), [meta](<https://devfeed.tech/tags/meta.md>), [model](<https://devfeed.tech/tags/model.md>), [native](<https://devfeed.tech/tags/native.md>), [open](<https://devfeed.tech/tags/open.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [open-source-collaboration](<https://devfeed.tech/tags/open-source-collaboration.md>), [pytorch](<https://devfeed.tech/tags/pytorch.md>), [source](<https://devfeed.tech/tags/source.md>), [tpu](<https://devfeed.tech/tags/tpu.md>)

### AI overview

TorchTPU is an open source Google-Meta effort to run PyTorch natively on Google TPUs, with eager execution, `torch.compile` through Dynamo, and distributed APIs as core design goals. A basic port can require little more than changing a device from `cuda` to `tpu`, though model shapes, sharding, and operators may need tuning for peak performance. Its `DeferAndFuse` eager mode groups operations and is reported as 50% or more faster than strict eager execution; compilation caches reduce repeat XLA compilation. The excerpt describes broad model and framework validation, while noting active development and limitations: bounded dynamism is not enabled by default, some GPU features or workload patterns need adaptation, and default "skinny collectives" can cost 30-40% end-to-end performance. Higher performance can require TPU-specific kernels and topology-aware sharding. Reported case studies include TorchTitan training and vLLM inference results, with performance depending on model, hardware, and optimizations.

### Source excerpt

TL;DR TorchTPU is an open source collaboration between Google and Meta that introduces a native and principled PyTorch backend for Google TPUs. It enables seamless out-of-the-box model execution and TPU... The post Introducing TorchTPU appeared first on PyTorch.

## Apple's big AI week ends with Mac + Muse AI supercomputers

DevFeed: [Apple's big AI week ends with Mac + Muse AI supercomputers](<https://devfeed.tech/articles/apple-s-big-ai-week-ends-with-mac-muse-ai-supercomputers-88522.md>)

Original publisher: [Read original article](<https://www.computerworld.com/article/4236527/apples-big-ai-week-ends-with-mac-muse-ai-supercomputers.html>)

Author: Jonny Evans

Published: 2026-10-16T15:27:05Z

Content type: news

Language: en

Sources: [Computerworld](<https://devfeed.tech/sources/computerworld.md>)

Topics: [MLX](<https://devfeed.tech/topics/mlx.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [rdma](<https://devfeed.tech/topics/rdma.md>), [Mixture of Experts (MoE)](<https://devfeed.tech/topics/mixture-of-experts-moe.md>)

Tags: [accelerators](<https://devfeed.tech/tags/accelerators.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-development](<https://devfeed.tech/tags/ai-development.md>), [apple](<https://devfeed.tech/tags/apple.md>), [apple-artificial-intelligence-computers-and-peripherals-it-operations-mac-vendors-and-providers](<https://devfeed.tech/tags/apple-artificial-intelligence-computers-and-peripherals-it-operations-mac-vendors-and-providers.md>), [apple-silicon](<https://devfeed.tech/tags/apple-silicon.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [computers-and-peripherals](<https://devfeed.tech/tags/computers-and-peripherals.md>), [edition-us](<https://devfeed.tech/tags/edition-us.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [it-operations](<https://devfeed.tech/tags/it-operations.md>), [language-en](<https://devfeed.tech/tags/language-en.md>), [large-language-models-llms](<https://devfeed.tech/tags/large-language-models-llms.md>), [mac](<https://devfeed.tech/tags/mac.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [mlx](<https://devfeed.tech/tags/mlx.md>), [productivity](<https://devfeed.tech/tags/productivity.md>), [storytype-newsanalysis](<https://devfeed.tech/tags/storytype-newsanalysis.md>), [vendors-and-providers](<https://devfeed.tech/tags/vendors-and-providers.md>)

### AI overview

Apple's MLX update aims to make local large language model inference and fine-tuning more efficient on Apple silicon. It adds support for M5 GPU Neural Accelerators, low-latency RDMA communication across Macs, and more efficient mixture-of-experts implementations. The article also describes running Meta's Muse Spark Open model across Mac Studio systems, including reported on-device use for software development tasks.

### Source excerpt

< > Welcome to the week Apple showed us how serious it really is about artificial intelligence on Macs, with the company making more moves to deliver the tech in the most cost effective, responsible way. While it's true that today is iPhone Duo pre-order day, that's arguably not the big news. It is not the big news because Apple this week made huge steps in AI. Not only have its researchers come up with a nifty tool to turn Macs into distributed inferencing data centers, not only does the company now have a plan for Apple Intelligence in China, but it has also worked with Meta to make it possible to run on-premises instances of Muse Spark Open on a couple of Mac Studios. Its machine learning team also introduced a new version of MLX that enables AI researchers and developers to run large language models (LLMs) locally on Mac Studio with Apple silicon. What's new to MLX from Apple? In case you'd missed it, MLX leverages the unified memory architecture of Apple silicon, which is invaluable if you want to build new inference and fine-tuning methods. Over the past year, the ML team has worked on "several key innovations" to make it easier and more efficient to "run state-of-the-art LLMs locally on recent Apple silicon." These improvements include: Software support for the Neural Accelerators in the GPU of the M5 family of chips Low-latency communication across several Macs using Remote Direct Memory Access (RDMA) over Thunderbolt Efficient implementations of recent popular architectures such as mixture of experts (MoE) The release leans heavily into what Apple's teams have been doing with Muse, and once again demonstrates how the very architecture of Apple silicon supports AI inferencing. That's no surprise to anybody at this point: not only have I been banging on that drum regularly, but almost anyone who is anyone in AI development and research already works on a Mac. LLM inference with MLX on Mac Studio A new white paper from Apple's machine learning team shares what

## Nvidia and Cisco unveil next-gen network interface built for the AI data center boom

DevFeed: [Nvidia and Cisco unveil next-gen network interface built for the AI data center boom](<https://devfeed.tech/articles/nvidia-and-cisco-unveil-next-gen-network-interface-built-for-the-ai-data-center-boom-88322.md>)

Original publisher: [Read original article](<https://www.techspot.com/news/114222-nvidia-cisco-unveil-next-gen-network-interface-built.html>)

Author: Alfonso Maruccia

Published: 2026-10-15T19:08:00Z

Content type: news

Language: en

Sources: [TechSpot](<https://devfeed.tech/sources/techspot.md>)

Topics: [Networking hardware](<https://devfeed.tech/topics/networking-hardware.md>), [AI Networking](<https://devfeed.tech/topics/ai-networking.md>), [rdma](<https://devfeed.tech/topics/rdma.md>), [Cisco](<https://devfeed.tech/topics/cisco.md>), [Nvidia](<https://devfeed.tech/topics/nvidia.md>)

Tags: [adapters](<https://devfeed.tech/tags/adapters.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-data-center](<https://devfeed.tech/tags/ai-data-center.md>), [bandwidth](<https://devfeed.tech/tags/bandwidth.md>), [cisco](<https://devfeed.tech/tags/cisco.md>), [connectors](<https://devfeed.tech/tags/connectors.md>), [data-center](<https://devfeed.tech/tags/data-center.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [industry](<https://devfeed.tech/tags/industry.md>), [infrastructure](<https://devfeed.tech/tags/infrastructure.md>), [memory](<https://devfeed.tech/tags/memory.md>), [network](<https://devfeed.tech/tags/network.md>), [network-interface](<https://devfeed.tech/tags/network-interface.md>), [networking](<https://devfeed.tech/tags/networking.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [silicon-photonics](<https://devfeed.tech/tags/silicon-photonics.md>), [transceiver](<https://devfeed.tech/tags/transceiver.md>), [usfp](<https://devfeed.tech/tags/usfp.md>)

### AI overview

Nvidia and Cisco have introduced the Ultra-Dense Small Form Factor Pluggable (USFP), an optical transceiver interface designed to increase data center bandwidth for AI workloads. The article says the first transceivers will support 16 full-duplex lanes and switches and SuperNIC adapters, with later development expected to expand lane counts toward a planned total capacity of 12.8 Tb/s. The interface is also designed to improve signal integrity and power efficiency.

### Source excerpt

GPU maker Nvidia and networking specialist Cisco have unveiled a new network interface called Ultra-Dense Small Form Factor Pluggable (USFP). The new optical transceiver is designed to significantly increase bandwidth in data centers, helping accommodate the enormous data demands of modern AI models and training workloads. Read Entire Article

## Local and Distributed Large LLM Inference with MLX and RDMA on Mac Studio

DevFeed: [Local and Distributed Large LLM Inference with MLX and RDMA on Mac Studio](<https://devfeed.tech/articles/local-and-distributed-large-llm-inference-with-mlx-and-rdma-on-mac-studio-88211.md>)

Original publisher: [Read original article](<https://machinelearning.apple.com/research/mlx-distributed-inference>)

Published: 2026-10-15T00:00:00Z

Content type: article

Language: en

Sources: [Apple Machine Learning Research](<https://devfeed.tech/sources/apple-machine-learning-research.md>)

Topics: [MLX](<https://devfeed.tech/topics/mlx.md>), [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [rdma](<https://devfeed.tech/topics/rdma.md>), [On-device AI](<https://devfeed.tech/topics/on-device-ai.md>)

Tags: [apple-silicon](<https://devfeed.tech/tags/apple-silicon.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [inference](<https://devfeed.tech/tags/inference.md>), [llm-inference](<https://devfeed.tech/tags/llm-inference.md>), [mac](<https://devfeed.tech/tags/mac.md>), [mac-studio](<https://devfeed.tech/tags/mac-studio.md>), [memory-architecture](<https://devfeed.tech/tags/memory-architecture.md>), [rdma](<https://devfeed.tech/tags/rdma.md>), [unified-memory-architecture](<https://devfeed.tech/tags/unified-memory-architecture.md>)

### AI overview

The article benchmarks local and distributed LLM inference using MLX on Apple silicon Mac Studio systems. It describes pipeline and tensor parallelization over RDMA via Thunderbolt, compares model performance across M3 Ultra and M5 Ultra machines, and explains how this setup can run large models locally for agentic AI tools.

### Source excerpt

MLX on Mac with Apple silicon lets AI researchers and developers experiment locally with the latest models and methods in a private environment. The open-source array framework leverages the unified memory architecture of Apple silicon, enabling anyone to run new inference and fine-tuning methods on their own hardware and explore other AI techniques. In a previous post, we highlighted the advantages of the M5 architecture and its GPU Neural Accelerators.

## Push-Based KV Transfer to Improve TTFT and TPOT for Disaggregated Inference in vLLM

DevFeed: [Push-Based KV Transfer to Improve TTFT and TPOT for Disaggregated Inference in vLLM](<https://devfeed.tech/articles/push-based-kv-transfer-to-improve-ttft-and-tpot-for-disaggregated-inference-in-vllm-87122.md>)

Original publisher: [Read original article](<https://vllm.ai/blog/2026-10-14-nixl-push-connector>)

Author: Sunita Nadampalli (AWS), Nicolò Lucchesi (Mistral)

Published: 2026-10-14T00:00:00Z

Content type: article

Language: en

Sources: [vLLM Blog](<https://devfeed.tech/sources/vllm-blog.md>)

Topics: [vllm](<https://devfeed.tech/topics/vllm.md>), [rdma](<https://devfeed.tech/topics/rdma.md>), [gpudirect rdma](<https://devfeed.tech/topics/gpudirect-rdma.md>), [AWS Trainium](<https://devfeed.tech/topics/aws-trainium.md>), [AWS Inferentia](<https://devfeed.tech/topics/aws-inferentia.md>), [quantization](<https://devfeed.tech/topics/quantization.md>)

Tags: [ai-engineering](<https://devfeed.tech/tags/ai-engineering.md>), [amazon-ec2](<https://devfeed.tech/tags/amazon-ec2.md>), [aws-ec2](<https://devfeed.tech/tags/aws-ec2.md>), [aws-trainium](<https://devfeed.tech/tags/aws-trainium.md>), [disaggregation](<https://devfeed.tech/tags/disaggregation.md>), [inference](<https://devfeed.tech/tags/inference.md>), [kv-cache](<https://devfeed.tech/tags/kv-cache.md>), [llm-inference](<https://devfeed.tech/tags/llm-inference.md>), [model-serving](<https://devfeed.tech/tags/model-serving.md>), [nixl](<https://devfeed.tech/tags/nixl.md>), [open-source-ai](<https://devfeed.tech/tags/open-source-ai.md>), [performance](<https://devfeed.tech/tags/performance.md>), [rdma](<https://devfeed.tech/tags/rdma.md>), [vllm](<https://devfeed.tech/tags/vllm.md>), [vllm-blog](<https://devfeed.tech/tags/vllm-blog.md>)

### AI overview

The article introduces a push-based NIXL connector for vLLM disaggregated inference. It overlaps KV-cache transfer setup with prefill, moving work off the decode instance's latency-critical path. In a Qwen3-32B benchmark on a one-prefill, one-decode deployment using Amazon EC2 P5en instances with EFA, push mode achieved up to 3x better time to first token and up to 30% better time per output token than pull mode; results vary by workload and configuration.

### Source excerpt

A push-based NIXL connector for vLLM disaggregated serving: the prefill instance writes KV directly into decode memory over RDMA, taking the router relay, decode-side allocation, and transfer setup off the TTFT critical path.

[Next page](<https://devfeed.tech/topics/rdma.md?cursor=WyIyMDI2LTEwLTE0VDAwOjAwOjAwKzAwOjAwIiwgIjg3Njc1YTIxLTNkODEtNDEzMi04YTdhLTk4MWIzNDU3NGE0MiJd>)