# CUDA

CUDA is NVIDIA's accelerated-computing platform, providing software that enables applications to use GPUs.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## A Runtime System for Mitigating Crossbar Port Contention on SM-Partitioned GPUs

DevFeed: [A Runtime System for Mitigating Crossbar Port Contention on SM-Partitioned GPUs](<https://devfeed.tech/articles/a-runtime-system-for-mitigating-crossbar-port-contention-on-sm-partitioned-gpus-142983.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.24458>)

Author: Ziwen Wang, Qi Luo, Kunlin Li, Changxi Liu, Dongsheng Wang, Yun Chen

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Distributed Computing](<https://devfeed.tech/sources/arxiv-distributed-computing.md>)

Topics: [GPU optimization](<https://devfeed.tech/topics/gpu-optimization.md>), [scheduling](<https://devfeed.tech/topics/scheduling.md>), [CUDA](<https://devfeed.tech/topics/cuda.md>)

Tags: [cache](<https://devfeed.tech/tags/cache.md>), [cs-ar](<https://devfeed.tech/tags/cs-ar.md>), [cs-dc](<https://devfeed.tech/tags/cs-dc.md>), [cuda](<https://devfeed.tech/tags/cuda.md>), [gpus](<https://devfeed.tech/tags/gpus.md>), [kernel](<https://devfeed.tech/tags/kernel.md>), [latency](<https://devfeed.tech/tags/latency.md>), [memory-bandwidth](<https://devfeed.tech/tags/memory-bandwidth.md>), [port](<https://devfeed.tech/tags/port.md>), [profiling](<https://devfeed.tech/tags/profiling.md>), [runtime](<https://devfeed.tech/tags/runtime.md>), [sm](<https://devfeed.tech/tags/sm.md>), [system](<https://devfeed.tech/tags/system.md>), [throughput](<https://devfeed.tech/tags/throughput.md>)

### AI overview

GPCPlace mitigates crossbar-port contention that persists when CUDA Green Contexts partition GPU workloads across SMs. Spreading workloads across GPCs can make latency-critical requests queue behind background requests at shared ports; packing them into separate GPCs avoids that contention but reduces the ports available to latency-critical requests and can limit bandwidth. Since neither placement is consistently faster, GPCPlace profiles workloads on the target GPU to estimate slowdowns and dynamically chooses between spreading and packing. It hooks CUDA driver kernel launches to set SM masks without application changes. In 112 workload cases across 14 GPUs, it improved latency-critical performance over Green Contexts by up to 1.15x on average and 2.38x maximum, without degrading throughput.

### Source excerpt

arXiv:2611.24458v1 Announce Type: new Abstract: To avoid scheduling delays when running concurrent workloads on a GPU, Streaming Multiprocessor (SM) partitioning mechanisms such as CUDA Green Contexts restrict each workload to separate SMs. Despite this partitioning, concurrent workloads still significantly slow down latency-critical workloads, which recent studies attribute to shared L2 cache and memory contention. However, contention also occurs at the crossbar ports connecting each Graphics Processing Cluster (GPC) of SMs to the L2 cache, as all SMs in a GPC share one port. Green Contexts spread workloads across GPCs, placing concurrent workloads in the same GPCs and forcing latency-critical memory requests to queue behind background requests. Packing workloads into separate GPCs avoids this contention, but forces latency-critical memory requests through fewer ports, limiting memory bandwidth. Consequently, neither spreading nor packing is always faster across workloads and GPUs. Based on these observations, we present GPCPlace, a runtime system that dynamically selects between spreading and packing by profiling workloads on the target GPU to estimate both slowdowns, and hooks kernel launches in the CUDA driver to set SM masks without application modifications. Across 14 GPUs and 112 workload cases, GPCPlace speeds up latency-critical workloads by up to 1.15x on average and 2.38x maximum over Green Contexts without degrading throughput.

## NVIDIA RTX Spark Compact Desktops Out Now

DevFeed: [NVIDIA RTX Spark Compact Desktops Out Now](<https://devfeed.tech/articles/nvidia-rtx-spark-compact-desktops-out-now-141203.md>)

Original publisher: [Read original article](<https://blogs.nvidia.com/blog/rtx-spark-desktop-local-ai-available-now/>)

Author: Justin Walker

Published: 2026-11-18T14:00:31Z

Content type: release

Language: en

Sources: [NVIDIA](<https://devfeed.tech/sources/nvidia-blog.md>)

Topics: [Local AI](<https://devfeed.tech/topics/local-ai.md>), [NVIDIA RTX](<https://devfeed.tech/topics/nvidia-rtx.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [DGX Spark](<https://devfeed.tech/topics/dgx-spark.md>), [DGX Station](<https://devfeed.tech/topics/dgx-station.md>), [CUDA](<https://devfeed.tech/topics/cuda.md>), [Nvidia](<https://devfeed.tech/topics/nvidia.md>), [3D](<https://devfeed.tech/topics/3d.md>), [NemoClaw](<https://devfeed.tech/topics/nemoclaw.md>), [Nemotron](<https://devfeed.tech/topics/nemotron.md>)

Tags: [adobe](<https://devfeed.tech/tags/adobe.md>), [adobe-illustrator](<https://devfeed.tech/tags/adobe-illustrator.md>), [agentic-ai](<https://devfeed.tech/tags/agentic-ai.md>), [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [desktops](<https://devfeed.tech/tags/desktops.md>), [dgx-spark](<https://devfeed.tech/tags/dgx-spark.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [generation](<https://devfeed.tech/tags/generation.md>), [image](<https://devfeed.tech/tags/image.md>), [local](<https://devfeed.tech/tags/local.md>), [local-ai](<https://devfeed.tech/tags/local-ai.md>), [local-ai-agents](<https://devfeed.tech/tags/local-ai-agents.md>), [local-models](<https://devfeed.tech/tags/local-models.md>), [memory](<https://devfeed.tech/tags/memory.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [nvidia-rtx](<https://devfeed.tech/tags/nvidia-rtx.md>), [nvidia-rtx-spark](<https://devfeed.tech/tags/nvidia-rtx-spark.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [rtx-spark](<https://devfeed.tech/tags/rtx-spark.md>), [video](<https://devfeed.tech/tags/video.md>), [video-generation](<https://devfeed.tech/tags/video-generation.md>), [windows](<https://devfeed.tech/tags/windows.md>)

### AI overview

ASUS, Dell and MSI RTX Spark compact desktops are available with a 20-core Grace CPU, Blackwell RTX GPU, 6,144 CUDA cores and up to 128GB unified memory. The source describes them as dedicated Windows PCs for local models, fine-tuning and always-on agents; local processing can keep code and files on the PC and avoid per-token cloud API charges for those tasks. It also highlights 10-gigabit Ethernet and claimed workloads including 120-billion-parameter models, 90GB+ 3D scenes and real-time 4:2:2 video editing. Software updates span native Windows on Arm Adobe apps, CUDA and WSL development workflows, Blender 5.3 viewport support for DLSS 4.5 Ray Reconstruction (requiring the latest NVIDIA driver), and ComfyUI video tools. NVIDIA Switchyard in Hermes can route routine tasks to local models and advanced reasoning to cloud models. The text also notes forthcoming mini PCs and Windows ML options, including experimental DirectX 12 support.

### Source excerpt

New NVIDIA RTX Spark compact desktops from ASUS, Dell and MSI are available today, bringing local AI agents, image and video generation, and model fine-tuning to a dedicated Windows PC. With up to 128GB of unified memory and NVIDIA's AI software stack, these systems provide room for larger local models alongside RTX-accelerated creative and development [...]

## Nvidia reveals Rubin HPC, a GPU with 50,000+ CUDA cores for scientific simulation

DevFeed: [Nvidia reveals Rubin HPC, a GPU with 50,000+ CUDA cores for scientific simulation](<https://devfeed.tech/articles/nvidia-reveals-rubin-hpc-a-gpu-with-50-000-cuda-cores-for-scientific-simulation-137868.md>)

Original publisher: [Read original article](<https://www.techspot.com/news/114583-nvidia-reveals-rubin-hpc-gpu-50000-cuda-cores.html>)

Author: Skye Jacobs

Published: 2026-11-17T15:46:00Z

Content type: news

Language: en

Sources: [TechSpot](<https://devfeed.tech/sources/techspot.md>)

Topics: [High-Performance Computing](<https://devfeed.tech/topics/high-performance-computing.md>), [CUDA-X](<https://devfeed.tech/topics/cuda-x.md>), [NVIDIA Vera Rubin](<https://devfeed.tech/topics/nvidia-vera-rubin.md>), [CUDA](<https://devfeed.tech/topics/cuda.md>), [NVLink](<https://devfeed.tech/topics/nvlink.md>)

Tags: [accelerated-computing](<https://devfeed.tech/tags/accelerated-computing.md>), [acceleration](<https://devfeed.tech/tags/acceleration.md>), [agentic](<https://devfeed.tech/tags/agentic.md>), [agentic-workflows](<https://devfeed.tech/tags/agentic-workflows.md>), [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-agents](<https://devfeed.tech/tags/ai-agents.md>), [algorithms](<https://devfeed.tech/tags/algorithms.md>), [amd](<https://devfeed.tech/tags/amd.md>), [analysis](<https://devfeed.tech/tags/analysis.md>), [analytics](<https://devfeed.tech/tags/analytics.md>), [bandwidth](<https://devfeed.tech/tags/bandwidth.md>), [cuda](<https://devfeed.tech/tags/cuda.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [hardware](<https://devfeed.tech/tags/hardware.md>), [hpc](<https://devfeed.tech/tags/hpc.md>), [multi-core](<https://devfeed.tech/tags/multi-core.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [rubin](<https://devfeed.tech/tags/rubin.md>), [rubin-gpu](<https://devfeed.tech/tags/rubin-gpu.md>), [rubin-hpc](<https://devfeed.tech/tags/rubin-hpc.md>), [science](<https://devfeed.tech/tags/science.md>), [simulation](<https://devfeed.tech/tags/simulation.md>)

### AI overview

Nvidia says its Rubin HPC GPU, announced for the Vera Rubin platform and expected in 2028, will have more than 50,000 CUDA cores and target scientific and engineering simulations used to test ideas generated by AI agents. The company says its "balanced design" scales data movement with compute, using sparse data acceleration for recurring routines such as sparse matrix operations, stencil computations, sorting and reductions. GPUs will connect through seventh-generation NVLink, and agents can use CUDA-X libraries with tuned algorithms. Key HPC specifications remain undisclosed: memory capacity and bandwidth, double-precision capability, interconnect bandwidth, and whether the chip uses multiple dies. Nvidia has not confirmed its physical layout.

### Source excerpt

The core count offers the first clue to what Nvidia is building. The current Rubin GPU has 224 streaming multiprocessors and 28,672 CUDA cores, so getting past 50,000 would take at least 74% more cores. Read Entire Article

## NVIDIA Announces Rubin HPC: Science-Focused Server GPU to Arrive in 2028

DevFeed: [NVIDIA Announces Rubin HPC: Science-Focused Server GPU to Arrive in 2028](<https://devfeed.tech/articles/nvidia-announces-rubin-hpc-science-focused-server-gpu-to-arrive-in-2028-140184.md>)

Original publisher: [Read original article](<https://www.servethehome.com/nvidia-announces-rubin-hpc-science-focused-server-gpu-to-arrive-in-2028/>)

Author: Ryan Smith

Published: 2026-11-17T15:45:19Z

Content type: news

Language: en

Sources: [ServeTheHome](<https://devfeed.tech/sources/servethehome.md>)

Topics: [High-Performance Computing](<https://devfeed.tech/topics/high-performance-computing.md>), [NVLink](<https://devfeed.tech/topics/nvlink.md>), [CUDA](<https://devfeed.tech/topics/cuda.md>), [Tensor Cores](<https://devfeed.tech/topics/tensor-cores.md>)

Tags: [accelerators](<https://devfeed.tech/tags/accelerators.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [hpc](<https://devfeed.tech/tags/hpc.md>), [news](<https://devfeed.tech/tags/news.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [rubin](<https://devfeed.tech/tags/rubin.md>), [rubin-hpc](<https://devfeed.tech/tags/rubin-hpc.md>), [sc26](<https://devfeed.tech/tags/sc26.md>), [science](<https://devfeed.tech/tags/science.md>), [server](<https://devfeed.tech/tags/server.md>)

### AI overview

NVIDIA says its Rubin HPC GPU, aimed at science and engineering workloads, is planned for release in 2028. The announced chip will have more than 50,000 CUDA cores--about twice as many as the current Rubin server GPU--reflecting an emphasis on vector processing that many traditional HPC workloads need. NVIDIA also cites a balanced approach to compute and data movement, NVLink 7, and "sparse data acceleration," but has not disclosed performance figures, memory details, or how the sparse-data feature works. Support across precisions from NVFP4 to FP64 is promised; actual FP64 performance will depend on the architecture, including how much hardware is allocated to FP64 and tensor operations. The article presents the announcement as NVIDIA's return to dedicated HPC after Blackwell, with AMD's MI430X expected in 2027 and Intel preparing a PCIe GPU D-Series.

### Source excerpt

Marking their return to the HPC market, at SC26 NVIDIA has announced the Rubin HPC chip. The science and engineering-focused GPU will have over 50,000 CUDA cores and will be released in 2028 The post NVIDIA Announces Rubin HPC: Science-Focused Server GPU to Arrive in 2028 appeared first on ServeTheHome.

## Ian Buck Kicks Off NVIDIA's Week at SC26 as Science Enters the Age of Agents

DevFeed: [Ian Buck Kicks Off NVIDIA's Week at SC26 as Science Enters the Age of Agents](<https://devfeed.tech/articles/ian-buck-kicks-off-nvidia-s-week-at-sc26-as-science-enters-the-age-of-agents-138205.md>)

Original publisher: [Read original article](<https://blogs.nvidia.com/blog/ian-buck-sc26/>)

Author: Brian Caulfield

Published: 2026-11-17T14:00:10Z

Content type: news

Language: en

Sources: [NVIDIA](<https://devfeed.tech/sources/nvidia-blog.md>)

Topics: [Nemotron](<https://devfeed.tech/topics/nemotron.md>), [High-Performance Computing](<https://devfeed.tech/topics/high-performance-computing.md>), [Local AI](<https://devfeed.tech/topics/local-ai.md>), [Nvidia](<https://devfeed.tech/topics/nvidia.md>), [DGX Spark](<https://devfeed.tech/topics/dgx-spark.md>), [NVIDIA Vera](<https://devfeed.tech/topics/nvidia-vera.md>), [CUDA](<https://devfeed.tech/topics/cuda.md>), [NVLink](<https://devfeed.tech/topics/nvlink.md>)

Tags: [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-agents](<https://devfeed.tech/tags/ai-agents.md>), [code](<https://devfeed.tech/tags/code.md>), [cpu](<https://devfeed.tech/tags/cpu.md>), [events](<https://devfeed.tech/tags/events.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [high-performance](<https://devfeed.tech/tags/high-performance.md>), [hpc](<https://devfeed.tech/tags/hpc.md>), [ian-buck](<https://devfeed.tech/tags/ian-buck.md>), [nemotron](<https://devfeed.tech/tags/nemotron.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [rubin](<https://devfeed.tech/tags/rubin.md>), [sc26](<https://devfeed.tech/tags/sc26.md>), [science](<https://devfeed.tech/tags/science.md>), [supercomputing](<https://devfeed.tech/tags/supercomputing.md>)

### AI overview

NVIDIA's SC26 announcements combine locally runnable science agents with new HPC hardware and quantum-computing work. Nemotron for Science pairs the Nemotron 3.5 Super VL open model with an open chemistry dataset and post-training recipes; researchers can run it on DGX Spark, a workstation or their own supercomputer. Buck cited an Argonne agent that planned and tested hypotheses, selected simulations and identified two binding sites, plus examples for GROMACS repair, telescope data plotting and GPU porting. The Nsight AI Porting Agent reportedly sped up a weather-model routine by more than 5x. NVIDIA also announced Rubin HPC, supporting FP64 to FP4 and expected in 2028, and Vera CPUs selected for a German climate supercomputer expected in early 2028. For quantum research, NVIDIA, IBM and Fermilab plan a co-design platform; Buck cited a CUDA-Q decoding algorithm that reduced a research computation from 12 weeks to 45 minutes. These are reported announcements and results from the excerpt.

### Source excerpt

NVIDIA's Ian Buck kicked off the company's week at SC26 in Chicago on Monday with new tools for AI agents, software that can plan experiments, run code and check results on researchers' own systems. In NVIDIA's special address, Buck announced Nemotron for Science, the Rubin HPC GPU and an expanded Vera CPU platform for high-performance [...]

## NVIDIA SC26: Rubin HPC GPU With 50,000 CUDA Cores Due in 2028, 192-CPU Vera Racks in Early 2027, and a 13,000x Quantum Decoder

DevFeed: [NVIDIA SC26: Rubin HPC GPU With 50,000 CUDA Cores Due in 2028, 192-CPU Vera Racks in Early 2027, and a 13,000x Quantum Decoder](<https://devfeed.tech/articles/nvidia-sc26-rubin-hpc-gpu-with-50-000-cuda-cores-due-in-2028-192-cpu-vera-racks-in-early-2027-and-a-13-000x-quantum-decoder-136081.md>)

Original publisher: [Read original article](<https://www.storagereview.com/news/nvidia-sc26-rubin-hpc-gpu-vera-cpu-hpc-rack-nemotron-for-science-quantum>)

Author: Brian Beeler

Published: 2026-11-16T16:49:16Z

Content type: news

Language: en

Sources: [StorageReview.com](<https://devfeed.tech/sources/storagereview-com.md>)

Topics: [High-Performance Computing](<https://devfeed.tech/topics/high-performance-computing.md>), [NVIDIA Vera](<https://devfeed.tech/topics/nvidia-vera.md>), [CUDA](<https://devfeed.tech/topics/cuda.md>), [CUDA-X](<https://devfeed.tech/topics/cuda-x.md>), [Quantum Computing](<https://devfeed.tech/topics/quantum-computing.md>), [NVIDIA Vera Rubin](<https://devfeed.tech/topics/nvidia-vera-rubin.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [computing](<https://devfeed.tech/tags/computing.md>), [cpu](<https://devfeed.tech/tags/cpu.md>), [cuda](<https://devfeed.tech/tags/cuda.md>), [decoder](<https://devfeed.tech/tags/decoder.md>), [enterprise](<https://devfeed.tech/tags/enterprise.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [hpc](<https://devfeed.tech/tags/hpc.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [quantum](<https://devfeed.tech/tags/quantum.md>), [rubin](<https://devfeed.tech/tags/rubin.md>), [rubin-hpc](<https://devfeed.tech/tags/rubin-hpc.md>), [sc26](<https://devfeed.tech/tags/sc26.md>), [vera](<https://devfeed.tech/tags/vera.md>)

### AI overview

NVIDIA's SC26 announcements span scientific computing, quantum systems, and AI networking. Rubin HPC, due in 2028, has more than 50,000 CUDA cores, sparse-data acceleration, and seventh-generation NVLink; NVIDIA has not disclosed its memory capacity, FP64 throughput, power, configurations, or comparisons with its current Rubin GPU. Vera HPC racks, expected from Dell, HPE, Lenovo, and Supermicro in early 2027, combine 192 CPUs with about 300TB of memory, 1PB of storage, and more than 200TB/s of peak aggregate memory bandwidth. NVIDIA reports performance and efficiency gains over unnamed x86 systems, but these vendor tests were not independently audited in the excerpt. Other announcements include the Nemotron for Science model package and CUDA-X libraries; a quantum codesign effort with IBM and Fermilab, alongside NVIDIA's claim that its TCBP-JOBS decoder is up to 13,000 times faster than existing algorithms; and Lambda's deployment of co-packaged optics switches across more than 10,000 GPUs. NVIDIA reports lower latency and temperature versus pluggable optics, with similar throughput.

### Source excerpt

NVIDIA has announced a new GPU for scientific computing at SC26. Rubin HPC is a member of the Vera Rubin platform that puts more than 50,000 CUDA cores on a single GPU, and NVIDIA expects it to be available in 2028. The rest of the company's SC26 slate covers a 192-CPU Vera rack for simulation The post NVIDIA SC26: Rubin HPC GPU With 50,000 CUDA Cores Due in 2028, 192-CPU Vera Racks in Early 2027, and a 13,000x Quantum Decoder appeared first on StorageReview.com.

## CUDA at 20: How the Platform Transformed Scientific Computing

DevFeed: [CUDA at 20: How the Platform Transformed Scientific Computing](<https://devfeed.tech/articles/cuda-at-20-how-the-platform-transformed-scientific-computing-136619.md>)

Original publisher: [Read original article](<https://blogs.nvidia.com/blog/20-years-cuda/>)

Author: Brett Murray

Published: 2026-11-16T16:00:11Z

Content type: article

Language: en

Sources: [NVIDIA](<https://devfeed.tech/sources/nvidia-blog.md>)

Topics: [CUDA](<https://devfeed.tech/topics/cuda.md>), [CUDA-X](<https://devfeed.tech/topics/cuda-x.md>), [High-Performance Computing](<https://devfeed.tech/topics/high-performance-computing.md>), [AI Models](<https://devfeed.tech/topics/ai-models.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [anniversary](<https://devfeed.tech/tags/anniversary.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [cuda](<https://devfeed.tech/tags/cuda.md>), [developer-program](<https://devfeed.tech/tags/developer-program.md>), [developers](<https://devfeed.tech/tags/developers.md>), [gpu-computing](<https://devfeed.tech/tags/gpu-computing.md>), [high-performance-computing](<https://devfeed.tech/tags/high-performance-computing.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [sc26](<https://devfeed.tech/tags/sc26.md>), [science](<https://devfeed.tech/tags/science.md>), [scientific-computing](<https://devfeed.tech/tags/scientific-computing.md>), [software](<https://devfeed.tech/tags/software.md>), [supercomputing](<https://devfeed.tech/tags/supercomputing.md>)

### AI overview

CUDA gave scientists and developers a familiar, high-level way to use GPUs beyond graphics, extending accelerated computing to science and simulation. The excerpt highlights Nanoscale Molecular Dynamics and Visual Molecular Dynamics in molecular research, CUDA-assisted work that helped resolve the HIV capsid's atomic structure, and AlexNet's 2012 ImageNet win after training on two NVIDIA GPUs. It also says the publicly released AlphaFold 2 uses CUDA for GPU execution. NVIDIA reports more than 11 million CUDA developers and 1,000+ CUDA-X libraries; these figures and examples are presented as evidence of the platform's reach, not as a full account of its impact.

### Source excerpt

This week at SC26, NVIDIA is marking two decades of CUDA by honoring the researchers, developers and engineers who helped make GPU computing a foundation for modern science and AI. CUDA's 20th anniversary presents an occasion to reflect across computing eras, from the people who wrote the very first CUDA programs to the AI developers [...]

## Faster Scientific Simulations with New FP64 Algorithms in NVIDIA cuBLAS

DevFeed: [Faster Scientific Simulations with New FP64 Algorithms in NVIDIA cuBLAS](<https://devfeed.tech/articles/faster-scientific-simulations-with-new-fp64-algorithms-in-nvidia-cublas-133565.md>)

Original publisher: [Read original article](<https://developer.nvidia.com/blog/faster-scientific-simulations-with-new-fp64-algorithms-in-nvidia-cublas/>)

Author: Cole Brower

Published: 2026-11-13T22:13:51Z

Content type: article

Language: en

Sources: [NVIDIA Developer](<https://devfeed.tech/sources/nvidia-developer.md>), [NVIDIA Technical Blog](<https://devfeed.tech/sources/nvidia-technical-blog.md>)

Topics: [Tensor Cores](<https://devfeed.tech/topics/tensor-cores.md>), [Algorithms](<https://devfeed.tech/topics/algorithms.md>), [CUDA](<https://devfeed.tech/topics/cuda.md>), [Computational Chemistry / Materials Science](<https://devfeed.tech/topics/computational-chemistry-materials-science.md>), [High-Performance Computing](<https://devfeed.tech/topics/high-performance-computing.md>), [Nvidia](<https://devfeed.tech/topics/nvidia.md>), [Materials science and engineering](<https://devfeed.tech/topics/materials-science-and-engineering.md>), [Supercomputing](<https://devfeed.tech/topics/supercomputing.md>), [RTX PRO](<https://devfeed.tech/topics/rtx-pro.md>)

Tags: [2](<https://devfeed.tech/tags/2.md>), [3](<https://devfeed.tech/tags/3.md>), [3m](<https://devfeed.tech/tags/3m.md>), [64-bit](<https://devfeed.tech/tags/64-bit.md>), [a100](<https://devfeed.tech/tags/a100.md>), [algorithms](<https://devfeed.tech/tags/algorithms.md>), [chemistry](<https://devfeed.tech/tags/chemistry.md>), [computing](<https://devfeed.tech/tags/computing.md>), [cublas](<https://devfeed.tech/tags/cublas.md>), [cuda-x](<https://devfeed.tech/tags/cuda-x.md>), [data-center-cloud](<https://devfeed.tech/tags/data-center-cloud.md>), [developer-tools-techniques](<https://devfeed.tech/tags/developer-tools-techniques.md>), [execution](<https://devfeed.tech/tags/execution.md>), [faster](<https://devfeed.tech/tags/faster.md>), [general](<https://devfeed.tech/tags/general.md>), [hpc-scientific-computing](<https://devfeed.tech/tags/hpc-scientific-computing.md>), [materials](<https://devfeed.tech/tags/materials.md>), [materials-science](<https://devfeed.tech/tags/materials-science.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [precision](<https://devfeed.tech/tags/precision.md>), [science](<https://devfeed.tech/tags/science.md>), [simulation-modeling-design](<https://devfeed.tech/tags/simulation-modeling-design.md>), [simulations](<https://devfeed.tech/tags/simulations.md>), [time](<https://devfeed.tech/tags/time.md>), [with](<https://devfeed.tech/tags/with.md>)

### AI overview

CUDA Toolkit 13.4.2 cuBLAS can accelerate double-precision DGEMM and complex ZGEMM on integer Tensor Cores while preserving FP64-equivalent accuracy. Ozaki-II uses the Chinese Remainder Theorem to reduce integer GEMMs relative to Ozaki-I; cuBLAS selects between them automatically and tiles residue processing to limit workspace. For complex GEMM, the 2M method reduces the work from four real GEMMs to two. ADP chooses precision at runtime; if its required mantissa width exceeds the default or user maximum, cuBLAS falls back to native FP64 instructions. Reported results include up to 5.5x DGEMM speedup on VR NVL72 and 3.9x on GB200, plus up to 5x ZGEMM speedup on GB200. Application studies report CCSD and BerkeleyGW gains, with results depending on GPU, workload, and precision mode.

### Source excerpt

Many scientific simulations, from molecular chemistry to materials science, spend the majority of their execution time computing high-precision dense general...

## CAFIN: CUDA-accelerated Adaptive Fisher Information Matrix Network Optimization via GPU-Native Kernel Fusion

DevFeed: [CAFIN: CUDA-accelerated Adaptive Fisher Information Matrix Network Optimization via GPU-Native Kernel Fusion](<https://devfeed.tech/articles/cafin-cuda-accelerated-adaptive-fisher-information-matrix-network-optimization-via-gpu-native-kernel-fusion-132026.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.14589>)

Author: Maryam Arshad, Muhammad Khuram Shahzad

Published: 2026-11-13T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [CUDA](<https://devfeed.tech/topics/cuda.md>), [GPU optimization](<https://devfeed.tech/topics/gpu-optimization.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>)

Tags: [accelerated](<https://devfeed.tech/tags/accelerated.md>), [adaptive](<https://devfeed.tech/tags/adaptive.md>), [cifar-10](<https://devfeed.tech/tags/cifar-10.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cuda](<https://devfeed.tech/tags/cuda.md>), [fusion](<https://devfeed.tech/tags/fusion.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [gpu-memory](<https://devfeed.tech/tags/gpu-memory.md>), [kernel](<https://devfeed.tech/tags/kernel.md>), [latency](<https://devfeed.tech/tags/latency.md>), [matrix](<https://devfeed.tech/tags/matrix.md>), [native](<https://devfeed.tech/tags/native.md>), [network-optimization](<https://devfeed.tech/tags/network-optimization.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [pytorch](<https://devfeed.tech/tags/pytorch.md>), [rtx](<https://devfeed.tech/tags/rtx.md>), [rtx-3060](<https://devfeed.tech/tags/rtx-3060.md>), [ti](<https://devfeed.tech/tags/ti.md>), [tiny](<https://devfeed.tech/tags/tiny.md>), [training](<https://devfeed.tech/tags/training.md>)

### AI overview

CAFIN speeds up AdaFisher by fusing its GPU work while preserving the optimizer's mathematical formulation. Fused CUDA kernels progressively combine FIM preconditioner construction, moving-average updates, gradient preconditioning, momentum, weight decay, and parameter updates; shared-memory reuse, register-resident computation, and in-place operations reduce memory traffic and intermediate allocations. In ResNet-18 experiments on CIFAR-10, CIFAR-100, and Tiny ImageNet using GTX 1660 Ti and RTX 3060 Laptop GPUs, it reduced isolated FIM construction latency by up to 78.4% and optimizer-step latency by 2.1-26.9%, and removed about 18.91 MB of intermediate allocations per step. Numerical results stayed within a predefined FP32 tolerance, with comparable convergence accuracy across the evaluated datasets and random seeds. These findings are limited to the reported models, datasets, and GPUs.

### Source excerpt

arXiv:2611.14589v1 Announce Type: cross Abstract: Second-order optimizers improve deep neural network training using curvature information, but their practical efficiency is limited by GPU execution overhead. AdaFisher reduces Fisher Information Matrix (FIM) approximation costs through diagonal Kronecker structures and exponential moving averages, yet its PyTorch implementation requires sequential CUDA kernel launches, intermediate tensor allocations, and repeated memory accesses. We introduce CAFIN, a GPU-native CUDA kernel fusion framework that improves AdaFisher execution efficiency without changing its optimization formulation. The proposed approach progressively integrates FIM preconditioner construction, exponential moving average updates, gradient preconditioning, momentum accumulation, weight decay, and parameter updates into fused CUDA kernels. Shared-memory reuse, register-resident computation, and in-place operations further reduce memory traffic and intermediate allocations. Experiments using ResNet-18 on CIFAR-10, CIFAR-100, and Tiny ImageNet across NVIDIA GTX 1660 Ti and RTX 3060 Laptop GPUs demonstrate up to 78.4% lower isolated FIM construction latency and 2.1-26.9% lower optimizer-step latency. The implementation also eliminates approximately 18.91 MB of intermediate GPU memory allocations per optimizer step. Numerical validation confirms consistency within the predefined FP32 tolerance, while convergence accuracy remains comparable across evaluated datasets and random seeds. These findings demonstrate that CUDA kernel fusion can substantially accelerate second-order optimization while preserving AdaFisher's underlying mathematical formulation and learning behavior.

## MPPI-Generic: A CUDA Library for Stochastic Trajectory Optimization

DevFeed: [MPPI-Generic: A CUDA Library for Stochastic Trajectory Optimization](<https://devfeed.tech/articles/mppi-generic-a-cuda-library-for-stochastic-trajectory-optimization-131504.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2409.07563>)

Author: Bogdan Vlahov, Jason Gibson, Manan Gandhi, Evangelos A. Theodorou

Published: 2026-11-13T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Distributed Computing](<https://devfeed.tech/sources/arxiv-distributed-computing.md>)

Topics: [CUDA-X](<https://devfeed.tech/topics/cuda-x.md>), [CUDA](<https://devfeed.tech/topics/cuda.md>), [Library](<https://devfeed.tech/topics/library.md>), [generic programming](<https://devfeed.tech/topics/generic-programming.md>)

Tags: [accelerated](<https://devfeed.tech/tags/accelerated.md>), [algorithms](<https://devfeed.tech/tags/algorithms.md>), [api](<https://devfeed.tech/tags/api.md>), [c-plus-plus](<https://devfeed.tech/tags/c-plus-plus.md>), [cs-dc](<https://devfeed.tech/tags/cs-dc.md>), [cs-ms](<https://devfeed.tech/tags/cs-ms.md>), [cs-ro](<https://devfeed.tech/tags/cs-ro.md>), [cs-sy](<https://devfeed.tech/tags/cs-sy.md>), [cuda](<https://devfeed.tech/tags/cuda.md>), [eess-sy](<https://devfeed.tech/tags/eess-sy.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [gpu-accelerated](<https://devfeed.tech/tags/gpu-accelerated.md>), [library](<https://devfeed.tech/tags/library.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [performance](<https://devfeed.tech/tags/performance.md>), [real-time](<https://devfeed.tech/tags/real-time.md>), [robust](<https://devfeed.tech/tags/robust.md>), [trajectory-optimization](<https://devfeed.tech/tags/trajectory-optimization.md>), [tube](<https://devfeed.tech/tags/tube.md>)

### AI overview

MPPI-Generic is a C++/CUDA library for GPU-accelerated stochastic trajectory optimization, implementing Model Predictive Path Integral, Tube-Model Predictive Path Integral, and Robust Model Predictive Path Integral control. It supports existing dynamics models and cost functions, and lets researchers add their own through defined APIs without modifying the MPPI control code. The paper compares its computational performance with other MPPI implementations across multiple GPUs and reports real-time capabilities; the supplied abstract gives no specific benchmark results.

### Source excerpt

arXiv:2409.07563v5 Announce Type: replace-cross Abstract: This paper introduces a new C++/CUDA library for GPU-accelerated stochastic optimization called MPPI-Generic. It provides implementations of Model Predictive Path Integral control, Tube-Model Predictive Path Integral Control, and Robust Model Predictive Path Integral Control, and allows for these algorithms to be used across many pre-existing dynamics models and cost functions. Furthermore, researchers can create their own dynamics models or cost functions following our API definitions without needing to change the actual Model Predictive Path Integral Control code. Finally, we compare computational performance to other popular implementations of Model Predictive Path Integral Control over a variety of GPUs to show the real-time capabilities our library can allow for. Library code can be found at: https://acdslab.github.io/mppi-generic-website/ .

## With $23M in funding, Luminal is the latest startup to put Nvidia's CUDA moat in its crosshairs

DevFeed: [With $23M in funding, Luminal is the latest startup to put Nvidia's CUDA moat in its crosshairs](<https://devfeed.tech/articles/with-23m-in-funding-luminal-is-the-latest-startup-to-put-nvidia-s-cuda-moat-in-its-crosshairs-130458.md>)

Original publisher: [Read original article](<https://siliconangle.com/2026/11/12/with-23m-in-funding-luminal-is-the-latest-startup-to-put-nvidias-cuda-moat-in-its-crosshairs/>)

Author: Mike Wheatley

Published: 2026-11-12T15:00:44Z

Content type: news

Language: en

Sources: [SiliconANGLE](<https://devfeed.tech/sources/siliconangle.md>)

Topics: [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [CUDA](<https://devfeed.tech/topics/cuda.md>), [AWS Trainium](<https://devfeed.tech/topics/aws-trainium.md>), [Nvidia](<https://devfeed.tech/topics/nvidia.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [ai-chips](<https://devfeed.tech/tags/ai-chips.md>), [ai-inference](<https://devfeed.tech/tags/ai-inference.md>), [ai-infrastructure](<https://devfeed.tech/tags/ai-infrastructure.md>), [amd](<https://devfeed.tech/tags/amd.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [aws](<https://devfeed.tech/tags/aws.md>), [chip](<https://devfeed.tech/tags/chip.md>), [cloud-computing](<https://devfeed.tech/tags/cloud-computing.md>), [cuda](<https://devfeed.tech/tags/cuda.md>), [dalton-caldwell](<https://devfeed.tech/tags/dalton-caldwell.md>), [early-stage](<https://devfeed.tech/tags/early-stage.md>), [felicis](<https://devfeed.tech/tags/felicis.md>), [funding](<https://devfeed.tech/tags/funding.md>), [generative-ai](<https://devfeed.tech/tags/generative-ai.md>), [infra](<https://devfeed.tech/tags/infra.md>), [investors](<https://devfeed.tech/tags/investors.md>), [joe-fioti](<https://devfeed.tech/tags/joe-fioti.md>), [led](<https://devfeed.tech/tags/led.md>), [liquid](<https://devfeed.tech/tags/liquid.md>), [luminal](<https://devfeed.tech/tags/luminal.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [news](<https://devfeed.tech/tags/news.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [nvidia-corp](<https://devfeed.tech/tags/nvidia-corp.md>), [search-compiler](<https://devfeed.tech/tags/search-compiler.md>), [semiconductors](<https://devfeed.tech/tags/semiconductors.md>), [series](<https://devfeed.tech/tags/series.md>), [series-a](<https://devfeed.tech/tags/series-a.md>), [siliconangle](<https://devfeed.tech/tags/siliconangle.md>), [software](<https://devfeed.tech/tags/software.md>), [standard-capital](<https://devfeed.tech/tags/standard-capital.md>), [startup](<https://devfeed.tech/tags/startup.md>), [the-latest](<https://devfeed.tech/tags/the-latest.md>), [together-ai](<https://devfeed.tech/tags/together-ai.md>), [top-story-5](<https://devfeed.tech/tags/top-story-5.md>), [trainium](<https://devfeed.tech/tags/trainium.md>), [venture-capital](<https://devfeed.tech/tags/venture-capital.md>), [y-combinator](<https://devfeed.tech/tags/y-combinator.md>)

### AI overview

Luminal's $23 million Series A will support its effort to reduce the work required to run AI inference on chips outside Nvidia's CUDA ecosystem. Its search compiler benchmarks ways to execute a neural network on a target chip, generates code for the best-performing option, and caches it for recurring workloads; a managed inference cloud provides the runtime infrastructure. The company targets scalable, continuous inference applications and says it processes more than 45 trillion tokens monthly, with significant workloads running on AWS Trainium. The excerpt describes alternatives including Google's and AWS's own frameworks, Spectral Compute's toolkit for running CUDA-based code on other chips, and the open-source UXL Foundation's OneAPI. Luminal launched its compiler and cloud four months before the article; its planned expansion includes broader chip and model support.

### Source excerpt

Luminal AI Inc. has outlined a plan to tear down Nvidia Corp.'s formidable software moat in the artificial intelligence chip industry after raising $23 million in early-stage funding. Today's Series A round was led by Standard Capital and saw participation from existing investors such as Y Combinator, SagaVC, Liquid 2 and Felicis, bringing Luminal's total [...] The post With $23M in funding, Luminal is the latest startup to put Nvidia's CUDA moat in its crosshairs appeared first on SiliconANGLE.

## Fast Entropy Decoding for Sparse MVM on GPUs

DevFeed: [Fast Entropy Decoding for Sparse MVM on GPUs](<https://devfeed.tech/articles/fast-entropy-decoding-for-sparse-mvm-on-gpus-126683.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2603.01915>)

Author: Emil Sch\\"atzle, Tommaso Pegolotti, Markus P\\"uschel

Published: 2026-11-10T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Distributed Computing](<https://devfeed.tech/sources/arxiv-distributed-computing.md>)

Topics: [GPU optimization](<https://devfeed.tech/topics/gpu-optimization.md>), [GPU](<https://devfeed.tech/topics/gpu.md>), [CUDA](<https://devfeed.tech/topics/cuda.md>), [Open Source](<https://devfeed.tech/topics/open-source.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [ans](<https://devfeed.tech/tags/ans.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [c-plus-plus](<https://devfeed.tech/tags/c-plus-plus.md>), [compression](<https://devfeed.tech/tags/compression.md>), [cs-dc](<https://devfeed.tech/tags/cs-dc.md>), [cs-ms](<https://devfeed.tech/tags/cs-ms.md>), [cs-pf](<https://devfeed.tech/tags/cs-pf.md>), [cuda](<https://devfeed.tech/tags/cuda.md>), [decoding](<https://devfeed.tech/tags/decoding.md>), [entropy](<https://devfeed.tech/tags/entropy.md>), [fast](<https://devfeed.tech/tags/fast.md>), [gpus](<https://devfeed.tech/tags/gpus.md>), [matrix](<https://devfeed.tech/tags/matrix.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [parallel](<https://devfeed.tech/tags/parallel.md>), [speed](<https://devfeed.tech/tags/speed.md>), [times](<https://devfeed.tech/tags/times.md>)

### AI overview

The method applies dtANS, a lossless entropy coding technique designed for parallel GPU decoding, to sparse matrices in CSR format and decodes them alongside sparse matrix-vector multiplication (SpMVM). On SuiteSparse matrices with at least 2^15 entries and an average of at least 10 entries per row, it reduces storage versus the smallest of cuSPARSE's CSR, COO, and SELL formats in almost all cases, by up to 11.77x. For the majority of tested matrices with at least 2^25 nonzero entries, SpMVM is faster, with a best speedup of 3.48x. The reported comparison with AI-based multi-format AlphaSparse is limited by that system's extreme computation overhead. The authors provide open-source C++/CUDA headers with compression and multiplication kernels.

### Source excerpt

arXiv:2603.01915v3 Announce Type: cross Abstract: We present a novel, practical approach to speed up sparse matrix-vector multiplication (SpMVM) on GPUs. The novel key idea is to apply lossless entropy coding to further compress the sparse matrix when stored in one of the commonly supported formats. Our method is based on dtANS, our new lossless compression method that improves the entropy coding technique of asymmetric numeral systems (ANS) specifically for fast parallel GPU decoding when used in tandem with SpMVM. We apply dtANS on the widely used CSR format and present extensive benchmarks on the SuiteSparse collection of matrices against the state-of-the-art cuSPARSE library. On matrices with at least 2^(15) entries and at least 10 entries per row on average, our compression reduces the matrix size over the smallest cuSPARSE format (CSR, COO and SELL) in almost all cases and up to 11.77 times. Further, we achieve an SpMVM speedup for the majority of matrices with at least 2^(25) nonzero entries. The best speedup is 3.48x. We also show that we can improve over the AI-based multi-format AlphaSparse in an experiment that is limited due to its extreme computation overhead. We provide our code as an open source C++/CUDA header library, which includes both compression and multiplication kernels.

## Speed Up Image Generation with FLUX 3 Image Klein on NVIDIA Blackwell GPUs

DevFeed: [Speed Up Image Generation with FLUX 3 Image Klein on NVIDIA Blackwell GPUs](<https://devfeed.tech/articles/speed-up-image-generation-with-flux-3-image-klein-on-nvidia-blackwell-gpus-121823.md>)

Original publisher: [Read original article](<https://developer.nvidia.com/blog/speed-up-image-generation-with-flux-3-image-klein-on-nvidia-blackwell-gpus/>)

Author: Olivia Stoner

Published: 2026-11-06T22:20:20Z

Content type: tutorial

Language: en

Sources: [NVIDIA Developer](<https://devfeed.tech/sources/nvidia-developer.md>), [NVIDIA Technical Blog](<https://devfeed.tech/sources/nvidia-technical-blog.md>)

Topics: [flux](<https://devfeed.tech/topics/flux.md>), [TensorRT-LLM](<https://devfeed.tech/topics/tensorrt-llm.md>), [NVIDIA Blackwell](<https://devfeed.tech/topics/nvidia-blackwell.md>), [ai-image-generator](<https://devfeed.tech/topics/ai-image-generator.md>), [diffusion-transformers](<https://devfeed.tech/topics/diffusion-transformers.md>), [NVFP4](<https://devfeed.tech/topics/nvfp4.md>), [quantization](<https://devfeed.tech/topics/quantization.md>), [Multi-GPU](<https://devfeed.tech/topics/multi-gpu.md>), [CUDA](<https://devfeed.tech/topics/cuda.md>), [RTX PRO](<https://devfeed.tech/topics/rtx-pro.md>)

Tags: [agentic-ai-generative-ai](<https://devfeed.tech/tags/agentic-ai-generative-ai.md>), [content-creation-rendering](<https://devfeed.tech/tags/content-creation-rendering.md>), [cuda-graphs](<https://devfeed.tech/tags/cuda-graphs.md>), [diffusion-models](<https://devfeed.tech/tags/diffusion-models.md>), [flux-3-image](<https://devfeed.tech/tags/flux-3-image.md>), [image](<https://devfeed.tech/tags/image.md>), [image-generation](<https://devfeed.tech/tags/image-generation.md>), [latency](<https://devfeed.tech/tags/latency.md>), [nvfp4](<https://devfeed.tech/tags/nvfp4.md>), [nvidia-blackwell](<https://devfeed.tech/tags/nvidia-blackwell.md>)

### AI overview

TensorRT LLM VisualGen combines compilation, NVFP4, fused GEMM kernels, attention optimizations, and caching to accelerate FLUX 3 Image Klein on NVIDIA Blackwell GPUs. In 2048x2048 tests on one B200, the combined setup reduced Base latency from 22.36 to 6.46 seconds for text-to-image and from 54.64 to 12.28 seconds for image-to-image, up to 3.46x and 4.45x faster than uncompiled BF16. NVFP4 alone was up to 1.62x faster for text-to-image and 1.58x for image-to-image. Further options trade output similarity against speed: VC-Attention FP8 improved generation speed by 1.16-1.18x with reported LPIPS differences, while Sol-Attention and Cache-DiT reduced denoising work or reused intermediate outputs. Cache-DiT nearly halved Base text-to-image latency at tested resolutions, with smaller gains for the 16-step distilled model. Eight-GPU execution and CUDA Graphs provide additional gains, though non-denoising work limits some end-to-end scaling. The tutorial supplies YAML settings and commands for offline generation and serving; reported results focus on B200 and RTX PRO 6000 Blackwell GPUs.

### Source excerpt

When deploying diffusion models in design tools, image editors, and visual production workflows, developers must account for inference latency. Generating an...

## Characterizing CUDA Locality Domains on Blackwell GPUs

DevFeed: [Characterizing CUDA Locality Domains on Blackwell GPUs](<https://devfeed.tech/articles/characterizing-cuda-locality-domains-on-blackwell-gpus-121269.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.07535>)

Author: Rocky Wang, Joyce Zhou, Keith Jiang, Jason Zhao

Published: 2026-11-06T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Distributed Computing](<https://devfeed.tech/sources/arxiv-distributed-computing.md>)

Topics: [CUDA](<https://devfeed.tech/topics/cuda.md>), [GPU optimization](<https://devfeed.tech/topics/gpu-optimization.md>), [Multi-GPU](<https://devfeed.tech/topics/multi-gpu.md>), [NVIDIA Blackwell](<https://devfeed.tech/topics/nvidia-blackwell.md>), [NCCL](<https://devfeed.tech/topics/nccl.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [also](<https://devfeed.tech/tags/also.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [b300](<https://devfeed.tech/tags/b300.md>), [blackwell](<https://devfeed.tech/tags/blackwell.md>), [can](<https://devfeed.tech/tags/can.md>), [changes](<https://devfeed.tech/tags/changes.md>), [code](<https://devfeed.tech/tags/code.md>), [communication](<https://devfeed.tech/tags/communication.md>), [costs](<https://devfeed.tech/tags/costs.md>), [critical](<https://devfeed.tech/tags/critical.md>), [cross](<https://devfeed.tech/tags/cross.md>), [cs-dc](<https://devfeed.tech/tags/cs-dc.md>), [cuda](<https://devfeed.tech/tags/cuda.md>), [data](<https://devfeed.tech/tags/data.md>), [deepgemm](<https://devfeed.tech/tags/deepgemm.md>), [domain](<https://devfeed.tech/tags/domain.md>), [domains](<https://devfeed.tech/tags/domains.md>), [driver](<https://devfeed.tech/tags/driver.md>), [gpus](<https://devfeed.tech/tags/gpus.md>), [locality](<https://devfeed.tech/tags/locality.md>)

### AI overview

On one node with eight NVIDIA B300 GPUs, the study finds that CUDA 13.4 Locality Domains can reduce latency, but benefits depend on data access patterns. Die-aware adaptations reportedly improve NCCL 2.32.3 AllGather by up to 43.5%, nvbandwidth DtoD by 111.3%, and DeepGEMM EP by 37%. The abstract also reports allocation overhead for sizes above 32 MB, linked to barriers and TLB invalidation. The authors propose a GPU driver kernel module fix, submitted to NVIDIA, that they say could reduce this overhead 7-10x without user-code changes.

### Source excerpt

arXiv:2611.07535v1 Announce Type: new Abstract: Low latency is critical to user experience. As NVIDIA Blackwell and Rubin GPUs advance multi-die scaling, data placement and cross-die communication become increasingly important. However, the latency implications and underlying costs of cross-die communication and die-aware execution remain largely unexplored. In this report, we present a comprehensive evaluation of the Locality Domain (LD) feature introduced in CUDA 13.4. Conducted on a single node equipped with eight NVIDIA B300 GPUs, our study examines three aspects: memory management and access, computation, and communication, to identify the sources of latency and, more importantly, explore how LD can mitigate them. Our measurements show that the benefits of LD depend on data access patterns. Through die-aware adaptations and integration with widely used libraries and tools, we achieve speedups of up to 43.5% for NCCL 2.32.3 (AllGather), 111.3% for nvbandwidth (DtoD), and 37% for DeepGEMM (EP). We also identify non-negligible LD-related GPU memory allocation overhead when allocation sizes exceed 32 MB due to barrier and TLB invalidation, we propose a fix to the GPU driver kernel module (submitted to NVIDIA) and show that overhead could be reduced 7x ~ 10x without requiring any user-code changes.

## Performance Portability of Out-of-Kernel Optimizations on GPUs: A Molecular Docking Case Study

DevFeed: [Performance Portability of Out-of-Kernel Optimizations on GPUs: A Molecular Docking Case Study](<https://devfeed.tech/articles/performance-portability-of-out-of-kernel-optimizations-on-gpus-a-molecular-docking-case-study-119110.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.06207>)

Author: Gianmarco Accordi, Leonardo Beltrame, Theresa Pollinger, Jens Domke, Davide Gadioli, Gianluca Palermo

Published: 2026-11-05T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Distributed Computing](<https://devfeed.tech/sources/arxiv-distributed-computing.md>)

Topics: [GPU optimization](<https://devfeed.tech/topics/gpu-optimization.md>), [Distributed Systems & Parallel Computing](<https://devfeed.tech/topics/distributed-systems-parallel-computing.md>), [CUDA](<https://devfeed.tech/topics/cuda.md>)

Tags: [amd](<https://devfeed.tech/tags/amd.md>), [applications](<https://devfeed.tech/tags/applications.md>), [backend](<https://devfeed.tech/tags/backend.md>), [batching](<https://devfeed.tech/tags/batching.md>), [case-study](<https://devfeed.tech/tags/case-study.md>), [concurrency](<https://devfeed.tech/tags/concurrency.md>), [cs-dc](<https://devfeed.tech/tags/cs-dc.md>), [cs-pf](<https://devfeed.tech/tags/cs-pf.md>), [cuda](<https://devfeed.tech/tags/cuda.md>), [efficiency](<https://devfeed.tech/tags/efficiency.md>), [execution](<https://devfeed.tech/tags/execution.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [gpus](<https://devfeed.tech/tags/gpus.md>), [hip](<https://devfeed.tech/tags/hip.md>), [intel](<https://devfeed.tech/tags/intel.md>), [kernel](<https://devfeed.tech/tags/kernel.md>), [memory](<https://devfeed.tech/tags/memory.md>), [methodology](<https://devfeed.tech/tags/methodology.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [optimizations](<https://devfeed.tech/tags/optimizations.md>), [performance](<https://devfeed.tech/tags/performance.md>)

### AI overview

For irregular multi-kernel GPU workloads, tuning decisions outside kernels--batching, worker concurrency, and per-worker memory allocation--can improve throughput across platforms. The authors implement explicit resource bounds in muDock's manager/worker model, where host workers share a GPU through independent streams, and evaluate NVIDIA, AMD, and Intel GPUs with CUDA, HIP, and SYCL. The memory-bounded formulation beats the occupancy-guided baseline's peak throughput across all tested platform/backend pairs; fully tuning the methodology improves end-to-end throughput in every tested configuration, with a reported average speedup of 1.53x. The evidence provided is the abstract, so it does not specify per-platform results or tuning details.

### Source excerpt

arXiv:2611.06207v1 Announce Type: new Abstract: Achieving performance portability is challenging for irregular scientific applications built as repeated multi-kernel GPU pipelines, because throughput depends on out-of-kernel decisions as much as on kernel efficiency. Using molecular docking as a representative case study, we propose an out-of-kernel tuning methodology that treats batching policy, worker concurrency, and per-worker memory allocation as explicit optimization parameters, replacing backend-dependent occupancy-driven choices with a more portable execution strategy based on explicit resource bounds. We implement this methodology in the muDock mini-app under a manager/worker execution model where multiple host workers share a GPU through independent streams. We then evaluate the methodology on NVIDIA, AMD, and Intel GPUs across CUDA, HIP, and SYCL implementations. Across all evaluated platform/backend pairs, the memory-bounded formulation exceeds the peak throughput of the occupancy-guided baseline. The fully tuned methodology improves the end-to-end throughput on all tested configurations, achieving an average speedup of 1.53x over the baseline.

## NVIDIA CUDA 13.5 Released With Production Support For Vera Rubin

DevFeed: [NVIDIA CUDA 13.5 Released With Production Support For Vera Rubin](<https://devfeed.tech/articles/nvidia-cuda-13-5-released-with-production-support-for-vera-rubin-117814.md>)

Original publisher: [Read original article](<https://www.phoronix.com/news/NVIDIA-CUDA-13.5>)

Author: Michael Larabel

Published: 2026-11-04T21:11:41Z

Content type: release

Language: en

Sources: [Phoronix](<https://devfeed.tech/sources/phoronix.md>)

Topics: [CUDA](<https://devfeed.tech/topics/cuda.md>), [NVIDIA Vera Rubin](<https://devfeed.tech/topics/nvidia-vera-rubin.md>), [CUDA Tile](<https://devfeed.tech/topics/cuda-tile.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [cuda](<https://devfeed.tech/tags/cuda.md>), [cuda-tile](<https://devfeed.tech/tags/cuda-tile.md>), [desktop-linux](<https://devfeed.tech/tags/desktop-linux.md>), [developer](<https://devfeed.tech/tags/developer.md>), [general-availability](<https://devfeed.tech/tags/general-availability.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [hardware](<https://devfeed.tech/tags/hardware.md>), [improvements](<https://devfeed.tech/tags/improvements.md>), [inference](<https://devfeed.tech/tags/inference.md>), [interoperability](<https://devfeed.tech/tags/interoperability.md>), [libraries](<https://devfeed.tech/tags/libraries.md>), [linux-benchmarking](<https://devfeed.tech/tags/linux-benchmarking.md>), [linux-hardware-benchmarks](<https://devfeed.tech/tags/linux-hardware-benchmarks.md>), [linux-hardware-reviews](<https://devfeed.tech/tags/linux-hardware-reviews.md>), [linux-how-to](<https://devfeed.tech/tags/linux-how-to.md>), [linux-performance](<https://devfeed.tech/tags/linux-performance.md>), [linux-server-benchmarks](<https://devfeed.tech/tags/linux-server-benchmarks.md>), [management](<https://devfeed.tech/tags/management.md>), [memory](<https://devfeed.tech/tags/memory.md>), [multi-gpu](<https://devfeed.tech/tags/multi-gpu.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [open-source-graphics](<https://devfeed.tech/tags/open-source-graphics.md>), [performance](<https://devfeed.tech/tags/performance.md>), [phoronix](<https://devfeed.tech/tags/phoronix.md>), [phoronix-test-suite](<https://devfeed.tech/tags/phoronix-test-suite.md>), [profiling](<https://devfeed.tech/tags/profiling.md>), [release](<https://devfeed.tech/tags/release.md>), [rubin](<https://devfeed.tech/tags/rubin.md>), [ubuntu-benchmarks](<https://devfeed.tech/tags/ubuntu-benchmarks.md>), [ubuntu-hardware](<https://devfeed.tech/tags/ubuntu-hardware.md>), [vera-rubin](<https://devfeed.tech/tags/vera-rubin.md>)

### AI overview

CUDA 13.5 brings general availability support for Vera Rubin, extending preliminary support from earlier CUDA 13 releases, along with performance improvements for the hardware. The excerpt also highlights multi-GPU communication and CUDA library interoperability for CUDA Tile, batched memory copies for CUDA Python, new GPU resource-management capabilities, and Nsight profiling capabilities for AI training and inference. It does not specify implementation details or quantify the improvements.

### Source excerpt

Along with today's release of the NVIDIA 620.70.06 Linux driver, NVIDIA also released CUDA 13.5 that brings general availability (GA) support for Vera Rubin...

## NVIDIA CUDA 13.5 Brings NVIDIA Vera Rubin GA Support

DevFeed: [NVIDIA CUDA 13.5 Brings NVIDIA Vera Rubin GA Support](<https://devfeed.tech/articles/nvidia-cuda-13-5-brings-nvidia-vera-rubin-ga-support-117642.md>)

Original publisher: [Read original article](<https://developer.nvidia.com/blog/nvidia-cuda-13-5-brings-nvidia-vera-rubin-ga-support/>)

Author: Shara Tibken

Published: 2026-11-04T18:46:07Z

Content type: release

Language: en

Sources: [NVIDIA Developer](<https://devfeed.tech/sources/nvidia-developer.md>), [NVIDIA Technical Blog](<https://devfeed.tech/sources/nvidia-technical-blog.md>)

Topics: [CUDA](<https://devfeed.tech/topics/cuda.md>), [CUDA Tile](<https://devfeed.tech/topics/cuda-tile.md>), [NVIDIA Vera Rubin](<https://devfeed.tech/topics/nvidia-vera-rubin.md>), [Nvidia](<https://devfeed.tech/topics/nvidia.md>), [CUDA Graphs](<https://devfeed.tech/topics/cuda-graphs.md>), [Tool](<https://devfeed.tech/topics/tool.md>), [NVLink](<https://devfeed.tech/topics/nvlink.md>)

Tags: [2026](<https://devfeed.tech/tags/2026.md>), [64-bit](<https://devfeed.tech/tags/64-bit.md>), [accelerated-computing](<https://devfeed.tech/tags/accelerated-computing.md>), [acceleration](<https://devfeed.tech/tags/acceleration.md>), [accounting](<https://devfeed.tech/tags/accounting.md>), [agent-skill](<https://devfeed.tech/tags/agent-skill.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-coding-agents](<https://devfeed.tech/tags/ai-coding-agents.md>), [ai-workloads](<https://devfeed.tech/tags/ai-workloads.md>), [arm](<https://devfeed.tech/tags/arm.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [attention](<https://devfeed.tech/tags/attention.md>), [automatic](<https://devfeed.tech/tags/automatic.md>), [backend](<https://devfeed.tech/tags/backend.md>), [cuda](<https://devfeed.tech/tags/cuda.md>), [cuda-tile](<https://devfeed.tech/tags/cuda-tile.md>), [cutile](<https://devfeed.tech/tags/cutile.md>), [developer-tools-techniques](<https://devfeed.tech/tags/developer-tools-techniques.md>), [featured](<https://devfeed.tech/tags/featured.md>), [machine-learning-artificial-intelligence](<https://devfeed.tech/tags/machine-learning-artificial-intelligence.md>), [nsight-tools-compute](<https://devfeed.tech/tags/nsight-tools-compute.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [nvidia-vera-rubin](<https://devfeed.tech/tags/nvidia-vera-rubin.md>), [python](<https://devfeed.tech/tags/python.md>), [release](<https://devfeed.tech/tags/release.md>), [rubin](<https://devfeed.tech/tags/rubin.md>), [vera-rubin](<https://devfeed.tech/tags/vera-rubin.md>)

### AI overview

CUDA Toolkit 13.5 makes NVIDIA Vera Rubin support generally available and adds platform and workload controls across CUDA. CUDA 13.5 can partition all SMs across locality domains, with discovery mode reporting potentially unequal partition sizes. MPS v3 adds weighted timeslicing, borrowable SM capacity, cgroup routing, namespace assignments, and polling as a memory-accounting fallback; polling adds work during eviction decisions. CUDA Graphs can capture batched 1D copies, and CUDA Tile adds multi-GPU communication plus preview support for calling selected SIMT library functions; custom device-function calls remain planned. CUDA Python and CCCL add batch copies, graph updates, public CUB tuning, batched Top-K, reproducible scans, and reductions that can consume GPU-resident counts. The release also updates profiling and debugging tools, math libraries, compiler controls, PTX, and binary inspection. On Rubin, WDDM and MCDM are supported; TCC is not. The excerpt describes many enhancements but directs readers to release notes for full compatibility details.

### Source excerpt

As artificial intelligence and accelerated computing continue to push the boundaries of what's possible, the latest NVIDIA CUDA release arrives to unlock...

## GC-MoE: Overlapping Expert-Parallel Token Dispatch with FP8/FP4 Grouped GEMM via CUDA Green Contexts

DevFeed: [GC-MoE: Overlapping Expert-Parallel Token Dispatch with FP8/FP4 Grouped GEMM via CUDA Green Contexts](<https://devfeed.tech/articles/gc-moe-overlapping-expert-parallel-token-dispatch-with-fp8-fp4-grouped-gemm-via-cuda-green-contexts-116485.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.04222>)

Author: Yong Li, Kan Liu, Tao Lan

Published: 2026-11-04T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Distributed Computing](<https://devfeed.tech/sources/arxiv-distributed-computing.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Mixture of Experts (MoE)](<https://devfeed.tech/topics/mixture-of-experts-moe.md>), [CUDA](<https://devfeed.tech/topics/cuda.md>), [CUDA Graphs](<https://devfeed.tech/topics/cuda-graphs.md>), [quantization](<https://devfeed.tech/topics/quantization.md>), [synchronization](<https://devfeed.tech/topics/synchronization.md>)

Tags: [blackwell](<https://devfeed.tech/tags/blackwell.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-ar](<https://devfeed.tech/tags/cs-ar.md>), [cs-dc](<https://devfeed.tech/tags/cs-dc.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-pf](<https://devfeed.tech/tags/cs-pf.md>), [cuda](<https://devfeed.tech/tags/cuda.md>), [decode](<https://devfeed.tech/tags/decode.md>), [decoding](<https://devfeed.tech/tags/decoding.md>), [deepseek](<https://devfeed.tech/tags/deepseek.md>), [deepseek-v4-1-flash](<https://devfeed.tech/tags/deepseek-v4-1-flash.md>), [dispatch](<https://devfeed.tech/tags/dispatch.md>), [expert-parallelism](<https://devfeed.tech/tags/expert-parallelism.md>), [fp4](<https://devfeed.tech/tags/fp4.md>), [fp8](<https://devfeed.tech/tags/fp8.md>), [glm](<https://devfeed.tech/tags/glm.md>), [glm-5-2](<https://devfeed.tech/tags/glm-5-2.md>), [gpus](<https://devfeed.tech/tags/gpus.md>), [graph](<https://devfeed.tech/tags/graph.md>), [handshake](<https://devfeed.tech/tags/handshake.md>), [hopper](<https://devfeed.tech/tags/hopper.md>), [inference](<https://devfeed.tech/tags/inference.md>), [mixture-of-experts](<https://devfeed.tech/tags/mixture-of-experts.md>), [mixture-of-experts-moe](<https://devfeed.tech/tags/mixture-of-experts-moe.md>), [open-source](<https://devfeed.tech/tags/open-source.md>)

### AI overview

GC-MoE overlaps expert-parallel token dispatch and grouped GEMM by using CUDA Green Contexts to assign communication and compute to separate GPU partitions, each with its own kernel resources and synchronization. A concurrent CUDA Graph launch, token-ring handshake, and double-buffered receive counters enable early computation and avoid a trailing cross-rank barrier between replays. For computation, it uses operand-swapped block-scaled MMA on Blackwell and register-A WGMMA with in-register FP4 decoding on Hopper; fused SwiGLU, requantization, and optional shared experts are supported. In the reported tests, it beat Mega MoE at all 54 matched-accuracy points: average decode speedups were 1.188x on 8 H20 GPUs for DeepSeek-V4.1-Flash, and 1.106x on 8 B300 GPUs for GLM-5.2 and DeepSeek-V4.1-Flash (1.123x with fused shared experts). Dispatch used 20 of 148 B300 SMs.

### Source excerpt

arXiv:2611.04222v1 Announce Type: new Abstract: Under expert parallelism, mixture-of-experts (MoE) layers pay all-to-all token dispatch, grouped GEMMs, and a combine step in every layer. In decode each rank holds only 1-256 tokens: dispatch latency and weight traffic dominate while tensor-core utilization is inherently low. In fused operators such as DeepGEMM's Mega MoE, dispatch and computation overlap only by sharing every SM: dispatch warps compete with TMA/MMA and epilogue warps for registers, shared memory, and issue slots; every phase shares one CTA resource budget, synchronization spans the whole SM array, and every replay ends with a workspace-clean pass and a cross-rank barrier. We present GC-MoE, an open-source FP8/FP4 MoE inference operator for NVIDIA Hopper and Blackwell GPUs that uses CUDA Green Contexts to split the device into a small communication partition and a large compute partition. Each role gets its own kernel, resource budget, and synchronization domain, so dispatch and GEMM tune independently. A three-node CUDA Graph (four on Hopper with fused shared experts) launches the dispatch and grouped-GEMM kernels as concurrent roots; a token-ring handshake starts computation on the first filled ring blocks, and double-buffered receive counters remove the trailing cross-rank barrier between replays. The compute partition runs operand-swapped tcgen05 block-scaled MMA on Blackwell and register-A WGMMA with in-register FP4 decoding on Hopper, with fused SwiGLU, requantization, and optional shared experts. On 8 H20 GPUs GC-MoE accelerates DeepSeek-V4.1-Flash decode by 1.188x on average on both the routed and shared-expert suites (peaks 1.296x and 1.353x); on 8 B300 GPUs it accelerates GLM-5.2 and DeepSeek-V4.1-Flash by 1.106x (1.123x with fused shared experts, peak 1.209x), beating Mega MoE at all 54 measured points at matched accuracy. GC-MoE reserves only 20 of 148 SMs on B300 for dispatch.

## OctEx: Billion-Scale Vector Search by Precomputed Lattice Neighbor Lists on GPUs

DevFeed: [OctEx: Billion-Scale Vector Search by Precomputed Lattice Neighbor Lists on GPUs](<https://devfeed.tech/articles/octex-billion-scale-vector-search-by-precomputed-lattice-neighbor-lists-on-gpus-113815.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.02556>)

Author: Dongfang Zhao

Published: 2026-11-03T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Distributed Computing](<https://devfeed.tech/sources/arxiv-distributed-computing.md>)

Topics: [ann](<https://devfeed.tech/topics/ann.md>), [CUDA Graphs](<https://devfeed.tech/topics/cuda-graphs.md>), [CUDA](<https://devfeed.tech/topics/cuda.md>)

Tags: [approximate-nearest-neighbor](<https://devfeed.tech/tags/approximate-nearest-neighbor.md>), [approximate-nearest-neighbor-search](<https://devfeed.tech/tags/approximate-nearest-neighbor-search.md>), [cs-dc](<https://devfeed.tech/tags/cs-dc.md>), [cuda](<https://devfeed.tech/tags/cuda.md>), [cuda-graphs](<https://devfeed.tech/tags/cuda-graphs.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [deduplication](<https://devfeed.tech/tags/deduplication.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [gpu-memory](<https://devfeed.tech/tags/gpu-memory.md>), [gpus](<https://devfeed.tech/tags/gpus.md>), [grid](<https://devfeed.tech/tags/grid.md>), [hbm](<https://devfeed.tech/tags/hbm.md>), [host-memory](<https://devfeed.tech/tags/host-memory.md>), [index](<https://devfeed.tech/tags/index.md>), [indexes](<https://devfeed.tech/tags/indexes.md>), [int8](<https://devfeed.tech/tags/int8.md>), [lattice](<https://devfeed.tech/tags/lattice.md>), [scale](<https://devfeed.tech/tags/scale.md>), [search](<https://devfeed.tech/tags/search.md>), [vector](<https://devfeed.tech/tags/vector.md>), [vector-search](<https://devfeed.tech/tags/vector-search.md>)

### AI overview

OctEx aims to reduce the dependent search rounds that limit high-recall GPU vector indexes such as CAGRA. It stores hundreds of randomly projected copies of a dataset, grouping vectors into nearly round grid cells with precomputed lists of vectors nearest each cell center. A query reads lists for its cells in parallel, expands from its best hits in a fixed number of waves, ranks candidates with an int8 sketch, then performs one exact rerank. Batched searches run as CUDA graphs across a server's GPUs, with index parts placed in GPU HBM or host memory according to access frequency. The abstract reports that on four H100 GPUs at recall 0.95-0.995, OctEx was 2.0-46.4x faster than CAGRA on seven datasets up to 100 million vectors and 1.46-2.05x faster on Deep1B and SIFT1B; those billion-vector indexes were built in 44-47 minutes. It also reports 5.4-343x speedups over GGNN, PathWeaver, FlowANN, and cuVS IVF-PQ and IVF-RaBitQ.

### Source excerpt

arXiv:2611.02556v1 Announce Type: new Abstract: Approximate nearest-neighbor search on GPUs serves retrieval-augmented generation, recommendation, and deduplication over datasets of up to billions of vectors. The fastest GPU indexes at high recall, such as CAGRA, walk proximity graphs in chained rounds, each of which waits for the distances of the previous one. High recall therefore costs many dependent rounds per query. Our insight is to spend the HBM of the GPUs on precomputed neighbor lists, so that a query reads its candidates in one parallel step instead of a chain of rounds. This paper turns the above insight into a GPU vector search system, namely OctEx, which holds hundreds of randomly projected copies of the dataset in GPU memory. Every copy rounds its vectors to the nearest points of a regular grid whose cells are nearly round and stores for every populated cell the list of vectors nearest to its center. A query reads the lists of its cells in all copies at once and then, in a fixed number of waves, the precomputed neighbor lists of its best hits. OctEx ranks the union of all hits by an int8 sketch before one exact rerank. OctEx runs the whole search of a batch as CUDA graphs over all GPUs of a server and places every part of its index in HBM or host memory by its access frequency. On four H100 GPUs at recall 0.95 to 0.995, OctEx outperforms CAGRA by 2.0x to 46.4x on seven datasets of up to one hundred million vectors, and by 1.46x to 2.05x on the one billion vectors of Deep1B and SIFT1B, whose indexes it builds in 44 to 47 minutes. OctEx also outperforms GGNN, PathWeaver, FlowANN, and the cuVS inverted file indexes IVF-PQ and IVF-RaBitQ by 5.4x to 343x.

## CuPH: Parallel Cubical Persistent Homology via Level-Synchronous Component Tracking

DevFeed: [CuPH: Parallel Cubical Persistent Homology via Level-Synchronous Component Tracking](<https://devfeed.tech/articles/cuph-parallel-cubical-persistent-homology-via-level-synchronous-component-tracking-113789.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.01078>)

Author: Guangyu Meng, Pengfei Gu, Erin Wolf Chambers, Danny Z. Chen

Published: 2026-11-03T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Distributed Computing](<https://devfeed.tech/sources/arxiv-distributed-computing.md>)

Topics: [High-Performance Computing](<https://devfeed.tech/topics/high-performance-computing.md>), [CUDA](<https://devfeed.tech/topics/cuda.md>), [GPU optimization](<https://devfeed.tech/topics/gpu-optimization.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Image](<https://devfeed.tech/topics/image.md>), [MONAI](<https://devfeed.tech/topics/monai.md>)

Tags: [algorithm](<https://devfeed.tech/tags/algorithm.md>), [backend](<https://devfeed.tech/tags/backend.md>), [classification](<https://devfeed.tech/tags/classification.md>), [component](<https://devfeed.tech/tags/component.md>), [connectivity](<https://devfeed.tech/tags/connectivity.md>), [cs-cg](<https://devfeed.tech/tags/cs-cg.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-dc](<https://devfeed.tech/tags/cs-dc.md>), [cuda](<https://devfeed.tech/tags/cuda.md>), [deep](<https://devfeed.tech/tags/deep.md>), [deep-learning](<https://devfeed.tech/tags/deep-learning.md>), [drop-in](<https://devfeed.tech/tags/drop-in.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [homology](<https://devfeed.tech/tags/homology.md>), [image](<https://devfeed.tech/tags/image.md>), [lock-free](<https://devfeed.tech/tags/lock-free.md>), [parallel](<https://devfeed.tech/tags/parallel.md>), [persistent-homology](<https://devfeed.tech/tags/persistent-homology.md>), [topology](<https://devfeed.tech/tags/topology.md>), [tracking](<https://devfeed.tech/tags/tracking.md>), [training](<https://devfeed.tech/tags/training.md>), [union-find](<https://devfeed.tech/tags/union-find.md>)

### AI overview

CuPH computes cubical persistent homology for 2D images by bucketing pixels by filtration value, activating them in a level-synchronous sweep, and tracking components with lock-free union-find. This replaces sequential reduction over cubical cells, reducing parallel depth from Θ(n) to O(L log n), where n is the pixel count and L the number of intensity levels; a single CUDA kernel avoids launches between levels. The authors report diagrams identical to standard cubical PH for integer inputs and a stability bound for floating-point inputs. Against GPU baseline CMP, CuPH is 18.6-99.6x faster. As a drop-in backend, it speeds topology-aware classification and segmentation while preserving accuracy. For high-resolution images, full-image PH losses improve Dice by 2.2-4.6 points over patch-local PH and accelerate training 25-31x.

### Source excerpt

arXiv:2611.01078v1 Announce Type: cross Abstract: Persistent homology (PH) is a mathematically principled multi-scale topological descriptor of shape and connectivity in data. It is widely applied in image classification, segmentation, and shape analysis across medical and aerial imaging, providing structural information that pixel-wise deep learning may overlook. Standard cubical PH decomposes an image into cubical cells (vertices, edges, and squares) and reduces them sequentially in filtration order. This sequential reduction has $\Theta(n)$ parallel depth for an $n$-pixel image, limiting within-image GPU parallelism and constraining its use in topology-aware deep learning. We introduce CuPH, a GPU-based parallel algorithm for computing cubical PH on 2D images that operates directly on pixels rather than cubical cells. CuPH buckets pixels by filtration values, activates them with a level-synchronous sweep, and tracks connected components with a lock-free union-find method. We prove that CuPH produces persistence diagrams identical to standard cubical PH on integer inputs, with a stability bound for floating-point inputs. CuPH reduces parallel depth from $\Theta(n)$ to $O(L\log n)$ for an image with $L$ intensity levels. We implement the level sweep as a single CUDA kernel, eliminating cross-level kernel-launch overhead. Empirically, CuPH is $18.6$--$99.6\times$ faster than the existing GPU baseline (CMP). As a drop-in PH backend, CuPH accelerates topology-aware classification ($4.2$--$16.7\times$) and segmentation ($8.3$--$44\times$) while preserving accuracy. It also enables computing full-image PH losses on high-resolution images, improving segmentation Dice by $2.2$--$4.6$ points over patch-local PH baselines and accelerating training by $25$--$31\times$.

## Making Local LLMs Usable on 8GB GPUs

DevFeed: [Making Local LLMs Usable on 8GB GPUs](<https://devfeed.tech/articles/making-local-llms-usable-on-8gb-gpus-112832.md>)

Original publisher: [Read original article](<https://hackernoon.com/making-local-llms-usable-on-8gb-gpus?source=rss>)

Author: Naresh Waghela

Published: 2026-11-02T19:35:58Z

Content type: tutorial

Language: en

Sources: [HackerNoon](<https://devfeed.tech/sources/hackernoon.md>)

Topics: [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>), [LLM evaluation / benchmarking](<https://devfeed.tech/topics/llm-evaluation-benchmarking.md>), [Local AI](<https://devfeed.tech/topics/local-ai.md>), [Cache](<https://devfeed.tech/topics/cache.md>), [quantization](<https://devfeed.tech/topics/quantization.md>), [GPU optimization](<https://devfeed.tech/topics/gpu-optimization.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [CUDA](<https://devfeed.tech/topics/cuda.md>)

Tags: [4k](<https://devfeed.tech/tags/4k.md>), [8k](<https://devfeed.tech/tags/8k.md>), [application-design](<https://devfeed.tech/tags/application-design.md>), [attention](<https://devfeed.tech/tags/attention.md>), [batch](<https://devfeed.tech/tags/batch.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [c-plus-plus](<https://devfeed.tech/tags/c-plus-plus.md>), [cache](<https://devfeed.tech/tags/cache.md>), [chat-template](<https://devfeed.tech/tags/chat-template.md>), [circuit-based-noise-mitigation](<https://devfeed.tech/tags/circuit-based-noise-mitigation.md>), [cmake](<https://devfeed.tech/tags/cmake.md>), [flash](<https://devfeed.tech/tags/flash.md>), [gpus](<https://devfeed.tech/tags/gpus.md>), [hackernoon-top-story](<https://devfeed.tech/tags/hackernoon-top-story.md>), [inference](<https://devfeed.tech/tags/inference.md>), [kv-cache](<https://devfeed.tech/tags/kv-cache.md>), [latency](<https://devfeed.tech/tags/latency.md>), [llama-cpp](<https://devfeed.tech/tags/llama-cpp.md>), [llm-optimization](<https://devfeed.tech/tags/llm-optimization.md>), [local](<https://devfeed.tech/tags/local.md>), [local-llm](<https://devfeed.tech/tags/local-llm.md>), [model-quantization](<https://devfeed.tech/tags/model-quantization.md>), [multilingual-llm](<https://devfeed.tech/tags/multilingual-llm.md>), [python-benchmarking](<https://devfeed.tech/tags/python-benchmarking.md>), [quantization](<https://devfeed.tech/tags/quantization.md>)

### AI overview

The guide provides a reproducible way to evaluate Qwen2.5-7B-Instruct with llama.cpp on an 8GB GPU, while emphasizing that its memory figures are calculations and it reports no measured performance gains. Budget for the KV cache as well as model weights: the estimated FP16 cache payload grows by about 1.53 GiB when context increases from 4K to 32K tokens, excluding runtime buffers and other GPU use. Establish a fixed baseline, record model and engine versions, and use the included Python script to measure first-text delay, decoding speed, and total time over repeated requests. Change one setting at a time to compare Flash Attention, Q8_0 cache, context size, and batch size; confirm support and actual server settings in logs. Check answer quality alongside latency, since cache quantization may affect quality and can hurt latency in some short-context cases. Five runs are only an initial check, and short prompts do not establish long-document performance.

### Source excerpt

Learn how to optimize local LLM inference on 8GB GPUs using KV-cache quantization, Flash Attention, batching, and practical latency benchmarks.

## Meet qwenGPU: Qwen3.5-2B With 45 Compiled GPU Layers That Run CUDA

DevFeed: [Meet qwenGPU: Qwen3.5-2B With 45 Compiled GPU Layers That Run CUDA](<https://devfeed.tech/articles/meet-qwengpu-qwen3-5-2b-with-45-compiled-gpu-layers-that-run-cuda-112519.md>)

Original publisher: [Read original article](<https://www.marktechpost.com/2026/11/01/qwengpu-simt-gpu-compiled-into-qwen3-5-2b-weights-runs-cuda/>)

Author: Michal Sutter

Published: 2026-11-02T07:28:29Z

Content type: article

Language: en

Sources: [MarkTechPost](<https://devfeed.tech/sources/marktechpost.md>)

Topics: [CUDA](<https://devfeed.tech/topics/cuda.md>), [model architecture](<https://devfeed.tech/topics/model-architecture.md>), [Neural Network](<https://devfeed.tech/topics/neural-network.md>)

Tags: [2](<https://devfeed.tech/tags/2.md>), [2026](<https://devfeed.tech/tags/2026.md>), [3](<https://devfeed.tech/tags/3.md>), [32-bit](<https://devfeed.tech/tags/32-bit.md>), [about](<https://devfeed.tech/tags/about.md>), [advanced](<https://devfeed.tech/tags/advanced.md>), [agentic-ai](<https://devfeed.tech/tags/agentic-ai.md>), [ai-infrastructure](<https://devfeed.tech/tags/ai-infrastructure.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [cuda](<https://devfeed.tech/tags/cuda.md>), [editors-pick](<https://devfeed.tech/tags/editors-pick.md>), [first](<https://devfeed.tech/tags/first.md>), [for-devs](<https://devfeed.tech/tags/for-devs.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [language-model](<https://devfeed.tech/tags/language-model.md>), [large-language-model](<https://devfeed.tech/tags/large-language-model.md>), [launches](<https://devfeed.tech/tags/launches.md>), [layers](<https://devfeed.tech/tags/layers.md>), [memory](<https://devfeed.tech/tags/memory.md>), [model](<https://devfeed.tech/tags/model.md>), [new-releases](<https://devfeed.tech/tags/new-releases.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [post](<https://devfeed.tech/tags/post.md>), [qwen3-5](<https://devfeed.tech/tags/qwen3-5.md>), [run](<https://devfeed.tech/tags/run.md>), [simt](<https://devfeed.tech/tags/simt.md>), [staff](<https://devfeed.tech/tags/staff.md>), [state](<https://devfeed.tech/tags/state.md>), [technology](<https://devfeed.tech/tags/technology.md>), [using](<https://devfeed.tech/tags/using.md>), [with](<https://devfeed.tech/tags/with.md>)

### AI overview

qwenGPU embeds a 4-lane SIMT GPU in Qwen3.5-2B using 45 untrained layers whose weights are compiler-generated. Gated DeltaNet recurrent state stores the program, registers, and memory; for each launch, the host compiles CUDA to PTX and supplies inputs, while the model emits execution steps and results. The developer reports exact checked launches and a reference interpreter matching a Tesla T4 on 49 of 49 kernels. In sampled tasks, normal mode produced GPU-computed answers for 20 of 24 questions, while advanced mode wrote a correct CUDA kernel on 4 of 5; the README cautions that a 2B model's CUDA remains unreliable. Reported speed is 10-15 tokens per second on Apple MPS, making this a research demo. Building requires Python 3.11+, uv, just, about 16 GB free memory, and clang with the NVPTX backend for advanced mode; no prebuilt checkpoint is published. These results are developer-reported.

### Source excerpt

qwenGPU compiles a 4-lane SIMT GPU into Qwen3.5-2B as 45 untrained layers. The model writes every step of real CUDA launches itself, using Gated DeltaNet state as registers and memory. The post Meet qwenGPU: Qwen3.5-2B With 45 Compiled GPU Layers That Run CUDA appeared first on MarkTechPost.

## Migrating a Legacy Scientific Application with an AI Coding Agent: NICAM from Fortran to Kokkos

DevFeed: [Migrating a Legacy Scientific Application with an AI Coding Agent: NICAM from Fortran to Kokkos](<https://devfeed.tech/articles/migrating-a-legacy-scientific-application-with-an-ai-coding-agent-nicam-from-fortran-to-kokkos-111753.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.41313>)

Author: Xuanzhengbo Ren, Takashi Shimokawabe, Hisashi Yashiro

Published: 2026-11-02T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Distributed Computing](<https://devfeed.tech/sources/arxiv-distributed-computing.md>)

Topics: [Kokkos](<https://devfeed.tech/topics/kokkos.md>), [Fortran](<https://devfeed.tech/topics/fortran.md>), [Claude Code](<https://devfeed.tech/topics/claude-code.md>), [legacy systems](<https://devfeed.tech/topics/legacy-systems.md>), [CUDA](<https://devfeed.tech/topics/cuda.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [ai-coding](<https://devfeed.tech/tags/ai-coding.md>), [ai-coding-agent](<https://devfeed.tech/tags/ai-coding-agent.md>), [c-plus-plus](<https://devfeed.tech/tags/c-plus-plus.md>), [comparison](<https://devfeed.tech/tags/comparison.md>), [cs-dc](<https://devfeed.tech/tags/cs-dc.md>), [cuda](<https://devfeed.tech/tags/cuda.md>), [fortran](<https://devfeed.tech/tags/fortran.md>), [hip](<https://devfeed.tech/tags/hip.md>), [human-review](<https://devfeed.tech/tags/human-review.md>), [legacy](<https://devfeed.tech/tags/legacy.md>), [migration](<https://devfeed.tech/tags/migration.md>), [performance](<https://devfeed.tech/tags/performance.md>)

### AI overview

With Claude Code assistance and human review, the researchers ported NICAM's dynamical core and seven physics packages from about 140,000 lines of Fortran/OpenACC to 35,000 lines of C++/Kokkos. A 100-member ensemble comparison found the port statistically consistent with the original. Against the Fortran/OpenACC baseline on GH200, application speedups were 1.71x on GH200 and 1.54x on MI300A. The CUDA build's GPU kernel time was 1.38x faster on GH200; on MI300A, the HIP build reached 0.96x baseline kernel performance after peak-memory-bandwidth normalization. The reported results show application-level gains across both tested clusters, while normalized HIP kernel performance was near, but below, baseline.

### Source excerpt

arXiv:2610.41313v1 Announce Type: new Abstract: Migrating from Fortran to C++/Kokkos broadens access to hardware accelerators through backends such as CUDA and HIP while offering greater flexibility in software design. However, legacy Fortran applications often have large codebases, making manual porting time-consuming and costly. Recent advances in large language models (LLMs) and AI coding agents have made large-scale code migration increasingly feasible. In this study, we ported NICAM (Nonhydrostatic ICosahedral Atmospheric Model), a legacy Fortran climate model accelerated with OpenACC, to C++/Kokkos with the assistance of Claude Code. The agent translated the code directly from Fortran to C++/Kokkos under human review. Approximately 140,000 lines of Fortran were analyzed and ported into 35,000 lines of C++/Kokkos, covering the dynamical core and seven physics packages. A 100-member ensemble comparison with the original Fortran model showed that the port is statistically consistent with the original. Performance was evaluated on Miyabi-G, an NVIDIA GH200 cluster, and Sirius, an AMD MI300A cluster. Compared with the Fortran/OpenACC baseline on GH200, the port achieved application speedups of $1.71\times$ on GH200 and $1.54\times$ on MI300A. The CUDA build achieved a GPU kernel-time speedup of $1.38\times$ on GH200. After normalization by peak memory bandwidth, the HIP build on MI300A achieved $0.96\times$ the kernel performance of the Fortran/OpenACC baseline.

## CUDA Oxide 0.3 Vastly Improves Rust Support For NVIDIA CUDA Programming

DevFeed: [CUDA Oxide 0.3 Vastly Improves Rust Support For NVIDIA CUDA Programming](<https://devfeed.tech/articles/cuda-oxide-0-3-vastly-improves-rust-support-for-nvidia-cuda-programming-109832.md>)

Original publisher: [Read original article](<https://www.phoronix.com/news/CUDA-Oxide-0.3>)

Author: Michael Larabel

Published: 2026-10-31T09:50:00Z

Content type: release

Language: en

Sources: [Phoronix](<https://devfeed.tech/sources/phoronix.md>)

Topics: [CUDA](<https://devfeed.tech/topics/cuda.md>), [Rust](<https://devfeed.tech/topics/rust.md>), [rust-ai](<https://devfeed.tech/topics/rust-ai.md>), [Hopper](<https://devfeed.tech/topics/hopper.md>), [NVFP4](<https://devfeed.tech/topics/nvfp4.md>)

Tags: [blackwell](<https://devfeed.tech/tags/blackwell.md>), [compiler](<https://devfeed.tech/tags/compiler.md>), [cuda](<https://devfeed.tech/tags/cuda.md>), [cuda-oxide](<https://devfeed.tech/tags/cuda-oxide.md>), [cutile-rs](<https://devfeed.tech/tags/cutile-rs.md>), [cutlass](<https://devfeed.tech/tags/cutlass.md>), [debugging](<https://devfeed.tech/tags/debugging.md>), [desktop-linux](<https://devfeed.tech/tags/desktop-linux.md>), [diagnostics](<https://devfeed.tech/tags/diagnostics.md>), [experimental](<https://devfeed.tech/tags/experimental.md>), [gdb](<https://devfeed.tech/tags/gdb.md>), [gpus](<https://devfeed.tech/tags/gpus.md>), [hopper](<https://devfeed.tech/tags/hopper.md>), [kernel](<https://devfeed.tech/tags/kernel.md>), [linux-benchmarking](<https://devfeed.tech/tags/linux-benchmarking.md>), [linux-hardware-benchmarks](<https://devfeed.tech/tags/linux-hardware-benchmarks.md>), [linux-hardware-reviews](<https://devfeed.tech/tags/linux-hardware-reviews.md>), [linux-how-to](<https://devfeed.tech/tags/linux-how-to.md>), [linux-performance](<https://devfeed.tech/tags/linux-performance.md>), [linux-server-benchmarks](<https://devfeed.tech/tags/linux-server-benchmarks.md>), [models](<https://devfeed.tech/tags/models.md>), [monorepo](<https://devfeed.tech/tags/monorepo.md>), [nightly-builds](<https://devfeed.tech/tags/nightly-builds.md>), [nvfp4](<https://devfeed.tech/tags/nvfp4.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [open-source-graphics](<https://devfeed.tech/tags/open-source-graphics.md>), [phoronix](<https://devfeed.tech/tags/phoronix.md>), [phoronix-test-suite](<https://devfeed.tech/tags/phoronix-test-suite.md>), [programming](<https://devfeed.tech/tags/programming.md>), [rust](<https://devfeed.tech/tags/rust.md>), [ubuntu-benchmarks](<https://devfeed.tech/tags/ubuntu-benchmarks.md>), [ubuntu-hardware](<https://devfeed.tech/tags/ubuntu-hardware.md>)

### AI overview

CUDA Oxide 0.3 switches to stable Rust 1.99.1, removing the nightly-build requirement described for earlier versions, and unifies its host runtime with cutile-rs in the cuda-rust monorepo. It adds an experimental Rust implementation of CUTLASS CuTe, cute-rs, which has run Qwen3-8B at BF16 + NVFP4 and Qwen3-TTS on NVIDIA Blackwell GPUs. Debugging and diagnostics include full cuda-gdb DWARF support, in-kernel event tracing, and register and spill diagnostics. The release also improves testing and adds performance and compiler optimizations for Blackwell and Hopper GPUs.

### Source excerpt

NVIDIA software engineers released CUDA Oxide 0.3 for closing out October. CUDA Oxide as a reminder is their experimental Rust to CUDA compiler that has been seeing a lot of activity in recent months...

[Next page](<https://devfeed.tech/topics/cuda.md?cursor=WyIyMDI2LTEwLTMxVDA5OjUwOjAwKzAwOjAwIiwgIjJmYmZiODhlLTBlZGItNGY4ZS1hNzVhLTZjYTAwOWVkMTMxZCJd>)