# CUDA Tile

CUDA Tile is a tile-based GPU programming model targeting NVIDIA Tensor Cores, with tools and language support for writing optimized tiled GPU kernels.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## Linearized 2-Simplicial Attention

DevFeed: [Linearized 2-Simplicial Attention](<https://devfeed.tech/articles/linearized-2-simplicial-attention-142552.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2608.09307>)

Author: Aritra Das, Dhruman Gupta, Debayan Gupta

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Transformer](<https://devfeed.tech/topics/transformer.md>), [CUDA Tile](<https://devfeed.tech/topics/cuda-tile.md>), [scaling laws](<https://devfeed.tech/topics/scaling-laws.md>)

Tags: [attention](<https://devfeed.tech/tags/attention.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cubic](<https://devfeed.tech/tags/cubic.md>), [features](<https://devfeed.tech/tags/features.md>), [fixed](<https://devfeed.tech/tags/fixed.md>), [hybrid](<https://devfeed.tech/tags/hybrid.md>), [kernels](<https://devfeed.tech/tags/kernels.md>), [kimi](<https://devfeed.tech/tags/kimi.md>), [perplexity](<https://devfeed.tech/tags/perplexity.md>), [softmax](<https://devfeed.tech/tags/softmax.md>), [training](<https://devfeed.tech/tags/training.md>), [validation](<https://devfeed.tech/tags/validation.md>)

### AI overview

The method reduces 2-simplicial attention's sequence-length cost from cubic to quadratic by rewriting its trilinear score as a composite query-key inner product, approximating the key sum with positive random features, and keeping previous keys in a fixed-size state. The sum over second keys remains exact across all previous tokens, unlike windowed 2-simplicial attention. Custom Triton kernels combine the method with Kimi Delta Attention in a model without softmax attention. In a 15B-token FineWeb-Edu comparison against a KDA hybrid, validation loss fell from 2.35 to 2.32 and LAMBADA perplexity from 40 to 38.

### Source excerpt

arXiv:2608.09307v2 Announce Type: replace Abstract: We present a linearized form of 2-simplicial attention by rewriting the trilinear score as an inner product between a composite query and a key, so that the sum over keys takes the same form as ordinary softmax attention. We then approximate this sum with positive random features and store all previous keys in a fixed-size state, while the sum over second keys stays exact over all previous tokens. This reduces the cost from cubic to quadratic in sequence length, and both keys range over all previous tokens, which windowed 2-simplicial attention does not allow. We implement it with custom Triton kernels and combine it with Kimi Delta Attention to build a model with no softmax attention at all. With 15B equal training tokens on FineWeb-Edu, this model lowers validation loss from 2.35 to 2.32 over a KDA hybrid and lowers LAMBADA perplexity from 40 to 38.

## KernelMutate: Mutation-Guided Validation of Generated GPU Kernels

DevFeed: [KernelMutate: Mutation-Guided Validation of Generated GPU Kernels](<https://devfeed.tech/articles/kernelmutate-mutation-guided-validation-of-generated-gpu-kernels-131713.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.15441>)

Author: Ziyuan Li, Zihe Song, Shuxiao Xie, Mingfang Ji, Xiaofang Qi, Wei Yang, Tao Xie

Published: 2026-11-13T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Software Engineering](<https://devfeed.tech/sources/arxiv-software-engineering.md>)

Topics: [Test automation](<https://devfeed.tech/topics/test-automation.md>), [CUDA Tile](<https://devfeed.tech/topics/cuda-tile.md>), [quantization](<https://devfeed.tech/topics/quantization.md>)

Tags: [boundary](<https://devfeed.tech/tags/boundary.md>), [cost](<https://devfeed.tech/tags/cost.md>), [coverage](<https://devfeed.tech/tags/coverage.md>), [cs-se](<https://devfeed.tech/tags/cs-se.md>), [cuda](<https://devfeed.tech/tags/cuda.md>), [custom](<https://devfeed.tech/tags/custom.md>), [execution](<https://devfeed.tech/tags/execution.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [independent](<https://devfeed.tech/tags/independent.md>), [kernels](<https://devfeed.tech/tags/kernels.md>), [mutation](<https://devfeed.tech/tags/mutation.md>), [native](<https://devfeed.tech/tags/native.md>), [offline](<https://devfeed.tech/tags/offline.md>), [precision](<https://devfeed.tech/tags/precision.md>), [ranking](<https://devfeed.tech/tags/ranking.md>), [recommendations](<https://devfeed.tech/tags/recommendations.md>), [reliability](<https://devfeed.tech/tags/reliability.md>), [replacement](<https://devfeed.tech/tags/replacement.md>), [results](<https://devfeed.tech/tags/results.md>), [validation](<https://devfeed.tech/tags/validation.md>)

### AI overview

A five-draw differential-testing anchor accepts 548 of 838 CUDA candidates, yet the full validation suite finds native FP32 witnesses for 136 accepted candidates and dtype-only findings for 362 more, mostly low-precision limitations without established publisher guarantees. Source-guided testing reaches the same 692 candidates at 26.1% lower charged cost, but for 16 accepted candidates with native witnesses in the full suite it provides only dtype evidence. Context and boundary findings cover the same candidate set. Offline template matching favors global top-eight recommendations over class-conditioned lists (51 versus 48 of 75 probes), though conditional analyses narrow that advantage at smaller limits; these are not deployed-suite gains. The study argues validation should report witness domains and observation coverage alongside candidate yield and cost, with independent controls and replacement cases exposing numerical-policy, binding, and custom-execution boundaries.

### Source excerpt

arXiv:2611.15441v1 Announce Type: new Abstract: Generated GPU implementations are commonly accepted after a few differential tests, although replacement obligations may remain unchecked. We present KernelMutate, an empirical study of validation reliability using controlled faults, released implementations, and replacement tests. A five-draw anchor accepts 548 of 838 CUDA candidates. The full suite finds native FP32 witnesses for 136 accepted candidates and dtype-only findings for another 362, mostly low-precision execution limitations without established publisher guarantees. Source-guided testing identifies the full suite's 692 candidates at 26.1% lower charged cost, but yields only dtype evidence for 16 accepted candidates with native witnesses in the full suite. Independently observed context and boundary findings cover the same candidate set. Task-held-out template matching reveals differences in ranking and observation coverage: global top-eight recommendations match more probes than class-conditioned lists (51 versus 48 of 75), while conditional analyses restrict the apparent advantage at smaller limits. These template-matching results are offline measurements, not deployed-suite gains. Independent controls and replacement cases expose numerical-policy, binding, and custom-execution boundaries. Validation therefore needs to report witness domains and observation coverage alongside candidate yield and cost.

## Fast by Default: Compile-Time Heuristics for Helion Kernels

DevFeed: [Fast by Default: Compile-Time Heuristics for Helion Kernels](<https://devfeed.tech/articles/fast-by-default-compile-time-heuristics-for-helion-kernels-131024.md>)

Original publisher: [Read original article](<https://pytorch.org/blog/fast-by-default-compile-time-heuristics-for-helion-kernels/>)

Author: Caleb Min Sup Kim, Ethan Che, Jongsok Choi, Jason Ansel

Published: 2026-11-12T16:45:26Z

Content type: article

Language: en

Sources: [PyTorch](<https://devfeed.tech/sources/blog-pytorch.md>)

Topics: [kernels](<https://devfeed.tech/topics/kernels.md>), [CUDA Tile](<https://devfeed.tech/topics/cuda-tile.md>), [AWS Inferentia](<https://devfeed.tech/topics/aws-inferentia.md>)

Tags: [analysis](<https://devfeed.tech/tags/analysis.md>), [analyze](<https://devfeed.tech/tags/analyze.md>), [aot](<https://devfeed.tech/tags/aot.md>), [arithmetic](<https://devfeed.tech/tags/arithmetic.md>), [assembler](<https://devfeed.tech/tags/assembler.md>), [attention](<https://devfeed.tech/tags/attention.md>), [axis](<https://devfeed.tech/tags/axis.md>), [backend](<https://devfeed.tech/tags/backend.md>), [backends](<https://devfeed.tech/tags/backends.md>), [batch](<https://devfeed.tech/tags/batch.md>), [benchmarking](<https://devfeed.tech/tags/benchmarking.md>), [blackwell](<https://devfeed.tech/tags/blackwell.md>), [block](<https://devfeed.tech/tags/block.md>), [blog](<https://devfeed.tech/tags/blog.md>), [dataflow](<https://devfeed.tech/tags/dataflow.md>), [helion](<https://devfeed.tech/tags/helion.md>), [ir](<https://devfeed.tech/tags/ir.md>), [kernel](<https://devfeed.tech/tags/kernel.md>), [kernels](<https://devfeed.tech/tags/kernels.md>), [pytorch](<https://devfeed.tech/tags/pytorch.md>), [reduction](<https://devfeed.tech/tags/reduction.md>)

### AI overview

Helion's compile-time heuristics analyze kernel IR and dataflow to choose settings such as block sizes, warps, and pipeline stages without benchmarking candidates. They classify kernels as matmul, reduction, or pointwise and estimate register, shared-memory, and tensor-memory use to guide choices; unknown compile-time extents still fall back to Helion's prior default. On the reported NVIDIA benchmarks, the heuristic improved results over that default: 1.18x handwritten Flash Linear Attention performance, 1.27x vLLM CUDA performance for reduction kernels, and 1.09x torch.compile max-autotune on a reduction suite. It also provided strong initial configs for autotuning, though seeded and unseeded searches reached nearly equal final results. The estimates are approximate, results include a jagged HSTU attention regression, and the heuristics are enabled for H100 and B200 on Triton; other backends remain unevaluated.

### Source excerpt

TL;DR Helion now uses compile-time heuristics that analyze a kernel's IR and dataflow to pick fast settings without slow autotuning. It classifies the kernel as matmul, reduction, or pointwise and... The post Fast by Default: Compile-Time Heuristics for Helion Kernels appeared first on PyTorch.

## Correct First, Fast Second: What a Local 7B Code Model Can and Cannot Do for GPU Kernel Optimization

DevFeed: [Correct First, Fast Second: What a Local 7B Code Model Can and Cannot Do for GPU Kernel Optimization](<https://devfeed.tech/articles/correct-first-fast-second-what-a-local-7b-code-model-can-and-cannot-do-for-gpu-kernel-optimization-125509.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.12079>)

Author: Jui-Chien Tsou, Po-Yi Wu

Published: 2026-11-10T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [CUDA Tile](<https://devfeed.tech/topics/cuda-tile.md>), [Local AI](<https://devfeed.tech/topics/local-ai.md>), [Retrieval Augmented Generation (RAG)](<https://devfeed.tech/topics/retrieval-augmented-generation-rag.md>), [Parallel reasoning](<https://devfeed.tech/topics/parallel-reasoning.md>)

Tags: [2](<https://devfeed.tech/tags/2.md>), [3](<https://devfeed.tech/tags/3.md>), [abstract](<https://devfeed.tech/tags/abstract.md>), [agent](<https://devfeed.tech/tags/agent.md>), [agentic](<https://devfeed.tech/tags/agentic.md>), [all](<https://devfeed.tech/tags/all.md>), [ambition](<https://devfeed.tech/tags/ambition.md>), [analysis](<https://devfeed.tech/tags/analysis.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [best](<https://devfeed.tech/tags/best.md>), [but](<https://devfeed.tech/tags/but.md>), [can](<https://devfeed.tech/tags/can.md>), [code](<https://devfeed.tech/tags/code.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [fast](<https://devfeed.tech/tags/fast.md>), [first](<https://devfeed.tech/tags/first.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [kernel](<https://devfeed.tech/tags/kernel.md>), [local](<https://devfeed.tech/tags/local.md>), [model](<https://devfeed.tech/tags/model.md>), [optimization](<https://devfeed.tech/tags/optimization.md>)

### AI overview

On 30 KernelBench tasks evaluated on a V100, correctness was the main limit for Qwen2.5-Coder-7B-Instruct generating CUDA kernels: even correct kernels averaged 0.97-0.99x eager PyTorch speed, with wins limited to fused memory-bound epilogues. Profiling-guided prompting performed best overall (43% correct and 27% faster than baseline), while a multi-agent loop was the only approach to solve level-2 fusion tasks (3/10). A first, naively scaffolded agent reached only 3% correctness, below zero-shot prompting's 30%. The reported improvements came from aligning agent ambition with model capability and using best-of-n sampling with an execution oracle: correctness rose from 3% at n=1 to 33% at n=8.

### Source excerpt

arXiv:2611.12079v1 Announce Type: new Abstract: We study how much the interaction strategy around a fixed, locally hosted 7B-parameter code model (Qwen2.5-Coder-7B-Instruct) affects its ability to replace PyTorch operators with custom CUDA kernels. On 30 KernelBench tasks (levels 1-3, first 10 each, evaluated on a V100), we compare four regimes of increasing structure: zero-shot prompting, profiling-guided prompting, iterative refinement on execution feedback, and a multi-agent loop (analysis, retrieval, generation, evaluation, feedback triage) in which the same model fills four roles around a non-LLM evaluator. Three findings stand out. (1) Correctness, not speed, is the binding constraint: even correct generated kernels average 0.97-0.99x the speed of eager PyTorch, because a 7B model cannot out-write cuBLAS/cuDNN on dense GEMM and convolution; the only real wins are fused memory-bound epilogues. (2) The best strategy depends on difficulty: profiling-guided prompting is strongest overall (43% correct, 27% faster than baseline), but the agent loop is the only method that solves any level-2 fusion problem (3/10 vs. 0/10 for all others). (3) Naive agentic scaffolding is actively harmful: our first agent scored 3% correct, below zero-shot's 30%, and the decisive repairs were matching the agent's ambition to the model's competence and spending compute on best-of-n sampling under a real execution oracle (3% to 33% correct from n=1 to n=8), not on more reasoning steps.

## CLion 2026.3 Is Here

DevFeed: [CLion 2026.3 Is Here](<https://devfeed.tech/articles/clion-2026-3-is-here-119019.md>)

Original publisher: [Read original article](<https://blog.jetbrains.com/clion/2026/11/2026-3-release/>)

Author: Oleg Zinovyev

Published: 2026-11-05T11:27:05Z

Content type: release

Language: en

Sources: [JetBrains](<https://devfeed.tech/sources/the-jetbrains-blog.md>)

Topics: [CLion](<https://devfeed.tech/topics/clion.md>), [CUDA Tile](<https://devfeed.tech/topics/cuda-tile.md>), [CMake](<https://devfeed.tech/topics/cmake.md>), [Tool](<https://devfeed.tech/topics/tool.md>), [junie](<https://devfeed.tech/topics/junie.md>), [Claude Code](<https://devfeed.tech/topics/claude-code.md>), [GitHub Copilot](<https://devfeed.tech/topics/github-copilot.md>), [jetbrains-toolbox](<https://devfeed.tech/topics/jetbrains-toolbox.md>)

Tags: [2026](<https://devfeed.tech/tags/2026.md>), [2026-3](<https://devfeed.tech/tags/2026-3.md>), [agent](<https://devfeed.tech/tags/agent.md>), [agentic](<https://devfeed.tech/tags/agentic.md>), [agentic-ai](<https://devfeed.tech/tags/agentic-ai.md>), [agentic-development](<https://devfeed.tech/tags/agentic-development.md>), [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [air](<https://devfeed.tech/tags/air.md>), [c](<https://devfeed.tech/tags/c.md>), [c-plus-plus](<https://devfeed.tech/tags/c-plus-plus.md>), [clion](<https://devfeed.tech/tags/clion.md>), [cmake](<https://devfeed.tech/tags/cmake.md>), [cuda](<https://devfeed.tech/tags/cuda.md>), [cuda-tile](<https://devfeed.tech/tags/cuda-tile.md>), [dap](<https://devfeed.tech/tags/dap.md>), [debugging](<https://devfeed.tech/tags/debugging.md>), [download](<https://devfeed.tech/tags/download.md>), [embedded](<https://devfeed.tech/tags/embedded.md>), [ispc](<https://devfeed.tech/tags/ispc.md>), [jetbrains](<https://devfeed.tech/tags/jetbrains.md>), [jetbrains-air](<https://devfeed.tech/tags/jetbrains-air.md>), [news](<https://devfeed.tech/tags/news.md>), [releases](<https://devfeed.tech/tags/releases.md>), [trace32](<https://devfeed.tech/tags/trace32.md>)

### AI overview

CLion 2026.3 adds JetBrains Air for running and reviewing agents, plus agent-assisted workflows for debugging ARM Cortex-M hard faults and diagnosing failed CMake loads. Air supports multiple agents and existing provider subscriptions, API keys, and custom base URLs; free Junie Lite is included with a JetBrains Account. Embedded debugging adds RTOS tasks, queues, and timers to agent tools. Debug profiles can be edited as JSON, with a dedicated TRACE32 launch and attach profile and configurable DAP working directories and environment variables. Language support includes CUDA Tile C++ for C++20 projects using CUDA Toolkit 13.3 or later, ISPC code insight, and automatic compiler headers and macros when no build targets provide settings. The release also adds class memory layout visualization and copies local project settings into Git worktrees. The Classic engine plugin's 2026.3 version is its last stable release; updates end in November 2026.

### Source excerpt

With JetBrains Air, agent-assisted embedded debugging and CMake project setup, plus support for NVIDIA CUDA Tile C++ and ISPC. CLion 2026.3 is now available. This latest version of the JetBrains IDE for C and C++ includes the following key features: You can download CLion 2026.3 from the link below, via the Toolbox App, as a [...]

## NVIDIA CUDA 13.5 Released With Production Support For Vera Rubin

DevFeed: [NVIDIA CUDA 13.5 Released With Production Support For Vera Rubin](<https://devfeed.tech/articles/nvidia-cuda-13-5-released-with-production-support-for-vera-rubin-117814.md>)

Original publisher: [Read original article](<https://www.phoronix.com/news/NVIDIA-CUDA-13.5>)

Author: Michael Larabel

Published: 2026-11-04T21:11:41Z

Content type: release

Language: en

Sources: [Phoronix](<https://devfeed.tech/sources/phoronix.md>)

Topics: [CUDA](<https://devfeed.tech/topics/cuda.md>), [NVIDIA Vera Rubin](<https://devfeed.tech/topics/nvidia-vera-rubin.md>), [CUDA Tile](<https://devfeed.tech/topics/cuda-tile.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [cuda](<https://devfeed.tech/tags/cuda.md>), [cuda-tile](<https://devfeed.tech/tags/cuda-tile.md>), [desktop-linux](<https://devfeed.tech/tags/desktop-linux.md>), [developer](<https://devfeed.tech/tags/developer.md>), [general-availability](<https://devfeed.tech/tags/general-availability.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [hardware](<https://devfeed.tech/tags/hardware.md>), [improvements](<https://devfeed.tech/tags/improvements.md>), [inference](<https://devfeed.tech/tags/inference.md>), [interoperability](<https://devfeed.tech/tags/interoperability.md>), [libraries](<https://devfeed.tech/tags/libraries.md>), [linux-benchmarking](<https://devfeed.tech/tags/linux-benchmarking.md>), [linux-hardware-benchmarks](<https://devfeed.tech/tags/linux-hardware-benchmarks.md>), [linux-hardware-reviews](<https://devfeed.tech/tags/linux-hardware-reviews.md>), [linux-how-to](<https://devfeed.tech/tags/linux-how-to.md>), [linux-performance](<https://devfeed.tech/tags/linux-performance.md>), [linux-server-benchmarks](<https://devfeed.tech/tags/linux-server-benchmarks.md>), [management](<https://devfeed.tech/tags/management.md>), [memory](<https://devfeed.tech/tags/memory.md>), [multi-gpu](<https://devfeed.tech/tags/multi-gpu.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [open-source-graphics](<https://devfeed.tech/tags/open-source-graphics.md>), [performance](<https://devfeed.tech/tags/performance.md>), [phoronix](<https://devfeed.tech/tags/phoronix.md>), [phoronix-test-suite](<https://devfeed.tech/tags/phoronix-test-suite.md>), [profiling](<https://devfeed.tech/tags/profiling.md>), [release](<https://devfeed.tech/tags/release.md>), [rubin](<https://devfeed.tech/tags/rubin.md>), [ubuntu-benchmarks](<https://devfeed.tech/tags/ubuntu-benchmarks.md>), [ubuntu-hardware](<https://devfeed.tech/tags/ubuntu-hardware.md>), [vera-rubin](<https://devfeed.tech/tags/vera-rubin.md>)

### AI overview

CUDA 13.5 brings general availability support for Vera Rubin, extending preliminary support from earlier CUDA 13 releases, along with performance improvements for the hardware. The excerpt also highlights multi-GPU communication and CUDA library interoperability for CUDA Tile, batched memory copies for CUDA Python, new GPU resource-management capabilities, and Nsight profiling capabilities for AI training and inference. It does not specify implementation details or quantify the improvements.

### Source excerpt

Along with today's release of the NVIDIA 620.70.06 Linux driver, NVIDIA also released CUDA 13.5 that brings general availability (GA) support for Vera Rubin...

## NVIDIA CUDA 13.5 Brings NVIDIA Vera Rubin GA Support

DevFeed: [NVIDIA CUDA 13.5 Brings NVIDIA Vera Rubin GA Support](<https://devfeed.tech/articles/nvidia-cuda-13-5-brings-nvidia-vera-rubin-ga-support-117642.md>)

Original publisher: [Read original article](<https://developer.nvidia.com/blog/nvidia-cuda-13-5-brings-nvidia-vera-rubin-ga-support/>)

Author: Shara Tibken

Published: 2026-11-04T18:46:07Z

Content type: release

Language: en

Sources: [NVIDIA Developer](<https://devfeed.tech/sources/nvidia-developer.md>), [NVIDIA Technical Blog](<https://devfeed.tech/sources/nvidia-technical-blog.md>)

Topics: [CUDA](<https://devfeed.tech/topics/cuda.md>), [CUDA Tile](<https://devfeed.tech/topics/cuda-tile.md>), [NVIDIA Vera Rubin](<https://devfeed.tech/topics/nvidia-vera-rubin.md>), [Nvidia](<https://devfeed.tech/topics/nvidia.md>), [CUDA Graphs](<https://devfeed.tech/topics/cuda-graphs.md>), [Tool](<https://devfeed.tech/topics/tool.md>), [NVLink](<https://devfeed.tech/topics/nvlink.md>)

Tags: [2026](<https://devfeed.tech/tags/2026.md>), [64-bit](<https://devfeed.tech/tags/64-bit.md>), [accelerated-computing](<https://devfeed.tech/tags/accelerated-computing.md>), [acceleration](<https://devfeed.tech/tags/acceleration.md>), [accounting](<https://devfeed.tech/tags/accounting.md>), [agent-skill](<https://devfeed.tech/tags/agent-skill.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-coding-agents](<https://devfeed.tech/tags/ai-coding-agents.md>), [ai-workloads](<https://devfeed.tech/tags/ai-workloads.md>), [arm](<https://devfeed.tech/tags/arm.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [attention](<https://devfeed.tech/tags/attention.md>), [automatic](<https://devfeed.tech/tags/automatic.md>), [backend](<https://devfeed.tech/tags/backend.md>), [cuda](<https://devfeed.tech/tags/cuda.md>), [cuda-tile](<https://devfeed.tech/tags/cuda-tile.md>), [cutile](<https://devfeed.tech/tags/cutile.md>), [developer-tools-techniques](<https://devfeed.tech/tags/developer-tools-techniques.md>), [featured](<https://devfeed.tech/tags/featured.md>), [machine-learning-artificial-intelligence](<https://devfeed.tech/tags/machine-learning-artificial-intelligence.md>), [nsight-tools-compute](<https://devfeed.tech/tags/nsight-tools-compute.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [nvidia-vera-rubin](<https://devfeed.tech/tags/nvidia-vera-rubin.md>), [python](<https://devfeed.tech/tags/python.md>), [release](<https://devfeed.tech/tags/release.md>), [rubin](<https://devfeed.tech/tags/rubin.md>), [vera-rubin](<https://devfeed.tech/tags/vera-rubin.md>)

### AI overview

CUDA Toolkit 13.5 makes NVIDIA Vera Rubin support generally available and adds platform and workload controls across CUDA. CUDA 13.5 can partition all SMs across locality domains, with discovery mode reporting potentially unequal partition sizes. MPS v3 adds weighted timeslicing, borrowable SM capacity, cgroup routing, namespace assignments, and polling as a memory-accounting fallback; polling adds work during eviction decisions. CUDA Graphs can capture batched 1D copies, and CUDA Tile adds multi-GPU communication plus preview support for calling selected SIMT library functions; custom device-function calls remain planned. CUDA Python and CCCL add batch copies, graph updates, public CUB tuning, batched Top-K, reproducible scans, and reductions that can consume GPU-resident counts. The release also updates profiling and debugging tools, math libraries, compiler controls, PTX, and binary inspection. On Rubin, WDDM and MCDM are supported; TCC is not. The excerpt describes many enhancements but directs readers to release notes for full compatibility details.

### Source excerpt

As artificial intelligence and accelerated computing continue to push the boundaries of what's possible, the latest NVIDIA CUDA release arrives to unlock...

## Ask, Don't Equip: Question-Driven Coding Agents for Deep Kernel Optimization

DevFeed: [Ask, Don't Equip: Question-Driven Coding Agents for Deep Kernel Optimization](<https://devfeed.tech/articles/ask-don-t-equip-question-driven-coding-agents-for-deep-kernel-optimization-114807.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.03517>)

Author: Gabriele Oliaro, Jaeseong Lee, Zhaoyuan Su, Sreeram Vennam, Zhihao Jia, Samyam Rajbhandari

Published: 2026-11-03T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [agentic workflows](<https://devfeed.tech/topics/agentic-workflows.md>), [Agent Harness](<https://devfeed.tech/topics/agent-harness.md>), [CUDA Tile](<https://devfeed.tech/topics/cuda-tile.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [agents](<https://devfeed.tech/tags/agents.md>), [analysis](<https://devfeed.tech/tags/analysis.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [autonomous](<https://devfeed.tech/tags/autonomous.md>), [blackwell](<https://devfeed.tech/tags/blackwell.md>), [coding](<https://devfeed.tech/tags/coding.md>), [coding-agents](<https://devfeed.tech/tags/coding-agents.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cublas](<https://devfeed.tech/tags/cublas.md>), [cuda](<https://devfeed.tech/tags/cuda.md>), [cudnn](<https://devfeed.tech/tags/cudnn.md>), [cutlass](<https://devfeed.tech/tags/cutlass.md>), [dependency](<https://devfeed.tech/tags/dependency.md>), [documentation](<https://devfeed.tech/tags/documentation.md>), [evaluations](<https://devfeed.tech/tags/evaluations.md>), [existing](<https://devfeed.tech/tags/existing.md>), [faster](<https://devfeed.tech/tags/faster.md>), [kernel](<https://devfeed.tech/tags/kernel.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [question](<https://devfeed.tech/tags/question.md>)

### AI overview

KernelSeek uses questions at decision points to direct CUDA kernel optimization without prescribing the next action: a general-purpose coding agent works within a minimal verification harness, first guided by a human and then by a question bank it uses itself. The authors argue this adds direction that scripted agents and open-ended search lack. On the same task, evaluator, hardware, and week, a tool-rich autonomous agent plateaued at 0.50x FlashAttention-3 after 133 steps, while KernelSeek reached 1.13x. It also produced dependency-free FlashAttention forward kernels for Hopper and Blackwell, correct across 182 scenarios per architecture, with geometric-mean speedups of 1.16x over FlashAttention-3 and 1.46x over FlashAttention-4. The abstract reports that every large gain followed a question that shifted the agent to another system layer or challenged a claimed limit, rather than supplying information.

### Source excerpt

arXiv:2611.03517v1 Announce Type: new Abstract: LLM agents for GPU kernel optimization sit at two ends of a structure-freedom tradeoff. Scripted agents fix what the model does at each step, via a learned policy, an optimization menu, or fixed roles, and are sample-efficient where the recipe applies and stall where it does not. Open-ended agents let a self-directed model mutate a population of kernels and pay for the freedom in trials: thousands of evaluations, repeated edits, and wrong candidates. Both respond to hard targets by equipping the agent with more: parsed profiler output, disassembly analysis, retrieval over documentation and libraries, and stores of lessons. We argue that both lack direction, a hypothesis about where the remaining gap is, and that a short question asked at the right moment supplies it without narrowing what the agent may do next. We present KernelSeek, a question-driven agent that makes an existing CUDA kernel faster with no operator-specific knowledge and no faster implementation to imitate: a general-purpose coding agent, a minimal harness that enforces a verifiable goal, and questions at decision points, first from a human, then from a bank the agent applies to itself. On the same task, evaluator, hardware, and week, a tool-rich autonomous agent built in the prevailing style plateaued at 0.50x FlashAttention-3 after 133 model-driven steps; KernelSeek reached 1.13x. It then produced dependency-free FlashAttention forward kernels for Hopper and Blackwell, single CUDA files with no CUTLASS, CuTe, cuBLAS, or cuDNN, correct on all 182 scenarios per architecture and 1.16x FlashAttention-3 and 1.46x FlashAttention-4 (geometric mean). Across 162 human messages and 11.5K agent tool calls, every large gain followed a question that moved the agent to a different layer of the system or rejected a claimed limit, never one that supplied information. We release the kernels, harness, question bank, and all traces.

## FlashBoB: I/O-Efficient Exact Backward-over-Backward for Softmax Attention

DevFeed: [FlashBoB: I/O-Efficient Exact Backward-over-Backward for Softmax Attention](<https://devfeed.tech/articles/flashbob-i-o-efficient-exact-backward-over-backward-for-softmax-attention-96874.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2609.24089>)

Author: Anthony Givans, Michael Crawshaw, Mingrui Liu

Published: 2026-10-23T04:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Transformer](<https://devfeed.tech/topics/transformer.md>), [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>), [CUDA Tile](<https://devfeed.tech/topics/cuda-tile.md>), [PSRAM](<https://devfeed.tech/topics/psram.md>), [Cache](<https://devfeed.tech/topics/cache.md>)

Tags: [algorithm](<https://devfeed.tech/tags/algorithm.md>), [attention](<https://devfeed.tech/tags/attention.md>), [bob](<https://devfeed.tech/tags/bob.md>), [context](<https://devfeed.tech/tags/context.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [deep-learning](<https://devfeed.tech/tags/deep-learning.md>), [gpu-memory](<https://devfeed.tech/tags/gpu-memory.md>), [gradient](<https://devfeed.tech/tags/gradient.md>), [high-bandwidth-memory](<https://devfeed.tech/tags/high-bandwidth-memory.md>), [i-o](<https://devfeed.tech/tags/i-o.md>), [lower-bound](<https://devfeed.tech/tags/lower-bound.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [order](<https://devfeed.tech/tags/order.md>), [pytorch](<https://devfeed.tech/tags/pytorch.md>), [sequence](<https://devfeed.tech/tags/sequence.md>), [sram](<https://devfeed.tech/tags/sram.md>), [times](<https://devfeed.tech/tags/times.md>), [transformer](<https://devfeed.tech/tags/transformer.md>)

### AI overview

FlashBoB makes exact backward-over-backward (BoB) for softmax attention more memory efficient by using a two-pass schedule based on a hierarchical affine structure in the double backward. It avoids all sequence-length-squared intermediate tensors, keeps computation within on-chip tiles, and uses bounded SRAM with minimal HBM traffic. In the stated score-recomputation model, its HBM traffic matches the inherited large-cache lower bound for exact forward attention. The abstract reports scaling to sequence length 262K on one A100 80GB GPU; prior PyTorch exact baselines fail by 16K, and FlashBoB is up to 6.3x faster than FlashBack. The evidence is the abstract, so it does not establish details beyond those claims.

### Source excerpt

arXiv:2609.24089v2 Announce Type: replace Abstract: Transformer models built on the attention mechanism have become a central building block in modern deep learning, yet softmax attention remains a major bottleneck for long-context workloads. While FlashAttention makes the forward and first backward passes I/O-efficient, it does not support backward-over-backward (BoB), which enables exact differentiation through the backward pass for applications such as second-order optimization, test-time training, gradient-based memory, and meta-learning. Existing BoB implementations either materialize large intermediate tensors or exhaust GPU memory at long sequence lengths. We present FlashBoB, an exact, I/O-efficient algorithm for BoB in softmax attention that keeps computation within on-chip tiles and avoids all $N \times N$ intermediate tensors, where $N$ is the sequence length. The key insight is a hierarchical affine structure in the softmax double backward: two row-wise scalars determine all outputs through affine transformations. This yields a two-pass schedule with bounded on-chip static random-access memory (SRAM) usage and minimal off-chip high-bandwidth memory (HBM) traffic. FlashBoB achieves $\Theta(N^2 d^2/M)$ HBM traffic ($d$ is the head dimension and $M$ is the memory size) and, within the standard FlashAttention-style score-recomputation model, matches the inherited large-cache lower bound for exact forward attention. Empirically, it scales exact attention BoB to $N=262\text{K}$ on a single A100 80GB GPU, where prior PyTorch exact baselines fail by $N=16\text{K}$, and is up to $6.3\times$ faster than FlashBack. These results make exact second-order attention practical at long-context sequence lengths where prior implementations cannot run efficiently.

## SAT Sampling at Scale via GPU-based Boolean Constraint Propagation

DevFeed: [SAT Sampling at Scale via GPU-based Boolean Constraint Propagation](<https://devfeed.tech/articles/sat-sampling-at-scale-via-gpu-based-boolean-constraint-propagation-95157.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.28805>)

Author: Alexander Nadel, Yogev Shalmon, Ofer Strichman

Published: 2026-10-22T04:00:00Z

Content type: article

Language: en

Sources: [arXiv: Distributed Computing](<https://devfeed.tech/sources/arxiv-distributed-computing.md>)

Topics: [Benchmark](<https://devfeed.tech/topics/benchmark.md>), [CUDA Tile](<https://devfeed.tech/topics/cuda-tile.md>), [Open Source](<https://devfeed.tech/topics/open-source.md>)

Tags: [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [boolean](<https://devfeed.tech/tags/boolean.md>), [concurrently](<https://devfeed.tech/tags/concurrently.md>), [cpu](<https://devfeed.tech/tags/cpu.md>), [cs-dc](<https://devfeed.tech/tags/cs-dc.md>), [cs-lo](<https://devfeed.tech/tags/cs-lo.md>), [faster](<https://devfeed.tech/tags/faster.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [gradient](<https://devfeed.tech/tags/gradient.md>), [hardware](<https://devfeed.tech/tags/hardware.md>), [library](<https://devfeed.tech/tags/library.md>), [parallelism](<https://devfeed.tech/tags/parallelism.md>), [quality](<https://devfeed.tech/tags/quality.md>), [random](<https://devfeed.tech/tags/random.md>), [sampling](<https://devfeed.tech/tags/sampling.md>), [scale](<https://devfeed.tech/tags/scale.md>), [throughput](<https://devfeed.tech/tags/throughput.md>), [verification](<https://devfeed.tech/tags/verification.md>)

### AI overview

Gpap-S combines SAT-style Boolean constraint propagation with GPU parallelism to sample solutions of Boolean formulas for constrained-random hardware verification. Its Gpap engine propagates thousands of independent partial assignments concurrently on a shared formula, aiming to retain SAT samplers' robustness at rare targets while increasing throughput. On benchmark families used to evaluate prior GPU gradient samplers, Gpap-S was 2.2-18.7x faster at the evaluated high-probability targets; at rare targets, where those samplers often produced no samples, it achieved roughly a 100x geometric-mean speedup. Across all benchmarks it was 83.4-1,571x faster than CPU-based CMSGen. Barbarik accepted both Gpap-S and CMSGen on every tested instance. The authors release Gpap as a reusable GPU library for high-throughput BCP.

### Source excerpt

arXiv:2610.28805v1 Announce Type: cross Abstract: Constrained-random verification tests hardware designs by sampling solutions of Boolean formulas. Existing samplers largely follow one of two approaches: CPU-based SAT samplers (e.g., CMSGen), which remain effective even at rare targets but offer limited throughput, and GPU-based gradient samplers, which achieve high throughput at high-probability targets but collapse at rare ones. We introduce Gpap-S, the first GPU sampler built on Boolean constraint propagation (BCP). At its core is Gpap, a new GPU engine for BCP that efficiently propagates many independent partial assignments concurrently on a shared Boolean formula. This capability allows Gpap-S to combine the rare-target robustness of SAT-based sampling with the throughput of GPU parallelism by exploring thousands of decide-and-propagate trajectories simultaneously. On the benchmark families used to evaluate prior GPU gradient samplers, Gpap-S is 2.2-18.7x faster at the high-probability targets considered in those evaluations. At rare targets, where those samplers often fail to generate any samples, Gpap-S achieves a geometric-mean speedup of roughly two orders of magnitude. Across all benchmarks, Gpap-S is also 83.4-1,571x faster than the CPU-based CMSGen. This throughput does not come at the expense of sample quality: under Barbarik, the statistical tester used to evaluate CMSGen, Gpap-S is accepted on every instance we test, as is CMSGen. We release Gpap as a reusable GPU library for high-throughput BCP.

## Trees from Marginals: Autoregressive drafting with factorized priors

DevFeed: [Trees from Marginals: Autoregressive drafting with factorized priors](<https://devfeed.tech/articles/trees-from-marginals-autoregressive-drafting-with-factorized-priors-94738.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2607.06763>)

Author: Yuma Oda, Ryan Mathieu, Roman Knyazhitskiy, Artur Chakhvadze

Published: 2026-10-22T04:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Model Optimizer](<https://devfeed.tech/topics/model-optimizer.md>), [CUDA Tile](<https://devfeed.tech/topics/cuda-tile.md>)

Tags: [acceptance](<https://devfeed.tech/tags/acceptance.md>), [adapter](<https://devfeed.tech/tags/adapter.md>), [algorithm](<https://devfeed.tech/tags/algorithm.md>), [autoregressive](<https://devfeed.tech/tags/autoregressive.md>), [conditional](<https://devfeed.tech/tags/conditional.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cuda](<https://devfeed.tech/tags/cuda.md>), [decoding](<https://devfeed.tech/tags/decoding.md>), [dependencies](<https://devfeed.tech/tags/dependencies.md>), [dflash](<https://devfeed.tech/tags/dflash.md>), [draft](<https://devfeed.tech/tags/draft.md>), [kernels](<https://devfeed.tech/tags/kernels.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [layers](<https://devfeed.tech/tags/layers.md>), [lightweight](<https://devfeed.tech/tags/lightweight.md>), [parallel](<https://devfeed.tech/tags/parallel.md>), [projection](<https://devfeed.tech/tags/projection.md>), [sglang](<https://devfeed.tech/tags/sglang.md>), [speculative-decoding](<https://devfeed.tech/tags/speculative-decoding.md>), [tokens](<https://devfeed.tech/tags/tokens.md>), [trees](<https://devfeed.tech/tags/trees.md>)

### AI overview

Weaver addresses falling acceptance rates in factorized speculative decoding by using a lightweight autoregressive adapter to build proposal trees from a factorized drafter's top-K token marginals, restoring dependencies among proposed tokens without a full-vocabulary projection. For models with Gated Delta Net layers, the authors also derive rollback-free tree verification and implement optimized CUDA kernels in SGLang. They report a 4.08x speedup over autoregressive decoding and 16.3% better performance than a highly optimized DFlash baseline.

### Source excerpt

arXiv:2607.06763v3 Announce Type: replace Abstract: Speculative decoding greatly increases the interactivity of autoregressive language models by trading off computation for extra tokens generated in a single forward pass. Factorized draft models are especially efficient because they predict future-token marginals in parallel, but their independence assumption causes acceptance rates to degrade sharply as the speculative budget grows. We analyze this limitation and introduce Weaver, a lightweight autoregressive adapter that constructs proposal trees from the top-K marginals of a factorized drafter. Weaver restores conditional dependencies between proposed tokens while avoiding a full-vocabulary projection. To support fast verification for models with Gated Delta Net layers, we derive a rollback-free tree-verification algorithm and implement optimized CUDA kernels in SGLang. By combining these model and systems contributions we achieve a 4.08-fold speedup over autoregressive decoding, and outperform a highly optimized DFlash baseline by 16.3%.

## TornadoVM 7 Enables GPU-Native Workloads From Java Applications

DevFeed: [TornadoVM 7 Enables GPU-Native Workloads From Java Applications](<https://devfeed.tech/articles/tornadovm-7-enables-gpu-native-workloads-from-java-applications-91286.md>)

Original publisher: [Read original article](<https://www.infoq.com/news/2026/10/tornadovm-7-gpu/>)

Author: Ben Evans

Published: 2026-10-21T03:00:00Z

Content type: release

Language: en

Sources: [InfoQ](<https://devfeed.tech/sources/infoq.md>)

Topics: [TornadoVM](<https://devfeed.tech/topics/tornadovm.md>), [CUDA](<https://devfeed.tech/topics/cuda.md>), [CUDA Tile](<https://devfeed.tech/topics/cuda-tile.md>), [Multi-GPU](<https://devfeed.tech/topics/multi-gpu.md>), [Java](<https://devfeed.tech/topics/java.md>), [OpenCL](<https://devfeed.tech/topics/opencl.md>), [openjdk](<https://devfeed.tech/topics/openjdk.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [ai-inference](<https://devfeed.tech/tags/ai-inference.md>), [ai-ml-data-engineering](<https://devfeed.tech/tags/ai-ml-data-engineering.md>), [apache-flink](<https://devfeed.tech/tags/apache-flink.md>), [apple](<https://devfeed.tech/tags/apple.md>), [architecture-design](<https://devfeed.tech/tags/architecture-design.md>), [cuda](<https://devfeed.tech/tags/cuda.md>), [cutile](<https://devfeed.tech/tags/cutile.md>), [development](<https://devfeed.tech/tags/development.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [java](<https://devfeed.tech/tags/java.md>), [llama3](<https://devfeed.tech/tags/llama3.md>), [ml-data-engineering](<https://devfeed.tech/tags/ml-data-engineering.md>), [news](<https://devfeed.tech/tags/news.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [tornadovm](<https://devfeed.tech/tags/tornadovm.md>), [tornadovm-7-gpu](<https://devfeed.tech/tags/tornadovm-7-gpu.md>)

### AI overview

TornadoVM 7.3 lets Java applications offload suitable data-parallel work to CPUs and GPUs while using mainstream JDKs. It manages accelerator memory and executes kernels, and supports local development with OpenCL or Apple Metal before deployment to Nvidia hardware. Version 7 replaces JNI with the Foreign Function & Memory API and removes the experimental JVMCI dependency; compiler metadata now follows one reflection, ASM, and Unsafe path across supported JDKs 21, 25, 26, and 27. Its optimized CUDA backend adds cuTile through TileContext, direct cuBLAS and CUTLASS calls in TaskGraphs, and multi-GPU execution. Developers can combine library calls with generated kernels to balance performance and portability. Similar hybrid support is described for Apple's MXL library. The excerpt cautions that offloading does not suit every computation; independent loop iterations are among the best candidates. It also describes a Flink integration that checks GPU availability and analyzes query IR for suitability before acceleration.

### Source excerpt

TornadoVM, an open-source heterogeneous programming framework for Java has reached version 7.0. It automatically accelerates Java programs on multi-core CPUs and GPUs and provides a rich GPU programming ecosystem in Java that can be used with any mainstream JDK distribution e.g. Temurin, Red Hat OpenJDK, Coretto, Oracle, GraalVM or Zulu. By Ben Evans

## Accelerating Mamba-2 with High-Performance Forward Pass Kernels in NVIDIA CUDA Tile

DevFeed: [Accelerating Mamba-2 with High-Performance Forward Pass Kernels in NVIDIA CUDA Tile](<https://devfeed.tech/articles/accelerating-mamba-2-with-high-performance-forward-pass-kernels-in-nvidia-cuda-tile-90421.md>)

Original publisher: [Read original article](<https://developer.nvidia.com/blog/accelerating-mamba-2-with-high-performance-forward-pass-kernels-in-nvidia-cuda-tile/>)

Author: Tanya Lenz

Published: 2026-10-19T17:00:00Z

Content type: article

Language: en

Sources: [NVIDIA Developer](<https://devfeed.tech/sources/nvidia-developer.md>), [NVIDIA Technical Blog](<https://devfeed.tech/sources/nvidia-technical-blog.md>)

Topics: [CUDA Tile](<https://devfeed.tech/topics/cuda-tile.md>), [CUDA](<https://devfeed.tech/topics/cuda.md>), [GPU optimization](<https://devfeed.tech/topics/gpu-optimization.md>), [High-Performance Computing](<https://devfeed.tech/topics/high-performance-computing.md>)

Tags: [benchmark](<https://devfeed.tech/tags/benchmark.md>), [cuda](<https://devfeed.tech/tags/cuda.md>), [cuda-tile](<https://devfeed.tech/tags/cuda-tile.md>), [cutile](<https://devfeed.tech/tags/cutile.md>), [data-science](<https://devfeed.tech/tags/data-science.md>), [developer-tools-techniques](<https://devfeed.tech/tags/developer-tools-techniques.md>), [dsl](<https://devfeed.tech/tags/dsl.md>), [gpus](<https://devfeed.tech/tags/gpus.md>), [high-performance](<https://devfeed.tech/tags/high-performance.md>), [kernels](<https://devfeed.tech/tags/kernels.md>), [mamba](<https://devfeed.tech/tags/mamba.md>), [mamba-2](<https://devfeed.tech/tags/mamba-2.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [parallel](<https://devfeed.tech/tags/parallel.md>), [performance](<https://devfeed.tech/tags/performance.md>), [programming](<https://devfeed.tech/tags/programming.md>), [python](<https://devfeed.tech/tags/python.md>)

### AI overview

The article explains how Jane Street and NVIDIA implemented Mamba-2 forward-pass kernels in CUDA Tile, a Python-based DSL for GPU programming. It walks through three kernels, describes optimizations including Tensor Core use, memory layout changes, and autotuning, and reports kernel-level speedups of 1.14x to 1.82x against a Triton implementation. The article notes that end-to-end training performance also depends on backward-pass implementations.

### Source excerpt

NVIDIA CUDA Tile (cuTile) is a Python-based DSL and parallel programming model for NVIDIA GPUs that automatically leverages advanced hardware capabilities to...

## One Search, Two Layers: Compiler Controls Inside PyTorch Helion using NVIDIA CompileIQ

DevFeed: [One Search, Two Layers: Compiler Controls Inside PyTorch Helion using NVIDIA CompileIQ](<https://devfeed.tech/articles/one-search-two-layers-compiler-controls-inside-pytorch-helion-using-nvidia-compileiq-90225.md>)

Original publisher: [Read original article](<https://pytorch.org/blog/one-search-two-layers-compiler-controls-inside-pytorch-helion-using-nvidia-compileiq/>)

Author: Aditya Srikanth (NVIDIA), Rupak Roy (NVIDIA), Abhinav Singh (NVIDIA), Jason Ansel (Meta), Clement Anthonioz-Blanc (Meta) and Jongsok Choi (Meta)

Published: 2026-10-19T15:45:05Z

Content type: tutorial

Language: en

Sources: [PyTorch](<https://devfeed.tech/sources/blog-pytorch.md>)

Topics: [Compiler](<https://devfeed.tech/topics/compiler.md>), [CUDA Tile](<https://devfeed.tech/topics/cuda-tile.md>), [NVFP4](<https://devfeed.tech/topics/nvfp4.md>), [PyTorch](<https://devfeed.tech/topics/pytorch.md>), [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>)

Tags: [blog](<https://devfeed.tech/tags/blog.md>), [blog-post](<https://devfeed.tech/tags/blog-post.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [helion](<https://devfeed.tech/tags/helion.md>), [kernels](<https://devfeed.tech/tags/kernels.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [python](<https://devfeed.tech/tags/python.md>), [pytorch](<https://devfeed.tech/tags/pytorch.md>)

### AI overview

The article explains how Helion can search NVIDIA PTXAS Advanced Control Files together with kernel configurations, allowing compiler settings such as register allocation and instruction scheduling to be tuned alongside tile sizes and warp counts. It describes using NVIDIA CompileIQ to generate or select ACFs, and reports measured GPU kernel speedups from tested workloads, while noting that gains depend on workload, hardware, and toolkit.

### Source excerpt

With Helion's release last year, it aimed at enabling developers to write GPU kernels in PyTorch-native Python and let its powerful autotuner do the work that used to require hand-written... The post One Search, Two Layers: Compiler Controls Inside PyTorch Helion using NVIDIA CompileIQ appeared first on PyTorch.

## Control How Your GPU Shares Work with Green Contexts

DevFeed: [Control How Your GPU Shares Work with Green Contexts](<https://devfeed.tech/articles/control-how-your-gpu-shares-work-with-green-contexts-65412.md>)

Original publisher: [Read original article](<https://developer.nvidia.com/blog/control-how-your-gpu-shares-work-with-green-contexts/>)

Author: Elizabeth Goodman

Published: 2026-10-06T15:00:00Z

Content type: tutorial

Language: en

Sources: [NVIDIA Developer](<https://devfeed.tech/sources/nvidia-developer.md>), [NVIDIA Technical Blog](<https://devfeed.tech/sources/nvidia-technical-blog.md>)

Topics: [Multi-GPU](<https://devfeed.tech/topics/multi-gpu.md>), [CUDA Tile](<https://devfeed.tech/topics/cuda-tile.md>), [Distributed Systems & Parallel Computing](<https://devfeed.tech/topics/distributed-systems-parallel-computing.md>), [NVLink](<https://devfeed.tech/topics/nvlink.md>)

Tags: [concurrency](<https://devfeed.tech/tags/concurrency.md>), [cuda](<https://devfeed.tech/tags/cuda.md>), [data-center-cloud](<https://devfeed.tech/tags/data-center-cloud.md>), [featured](<https://devfeed.tech/tags/featured.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [latency](<https://devfeed.tech/tags/latency.md>)

### AI overview

CUDA green contexts let applications assign GPU execution resources and workqueues to selected workloads, helping multiple workloads run concurrently with more predictable performance. The article explains the Runtime API setup and illustrates how partitioning can reduce latency for critical kernels while limiting resources available to bulk work.

### Source excerpt

GPU applications increasingly consist of multiple independent components running at the same time within a single process: a latency-sensitive operator...

## Optimizing Jagged Flash Attention with TLX: The Road Toward SOTA FA4 on Blackwell

DevFeed: [Optimizing Jagged Flash Attention with TLX: The Road Toward SOTA FA4 on Blackwell](<https://devfeed.tech/articles/optimizing-jagged-flash-attention-with-tlx-the-road-toward-sota-fa4-on-blackwell-63709.md>)

Original publisher: [Read original article](<https://pytorch.org/blog/optimizing-jagged-flash-attention-with-tlx-the-road-toward-sota-fa4-on-blackwell/>)

Author: Han Xu, Jacky Zhou, Jackie (Jiaqi) Xu, Hongtao Yu, Peng Chen (Dev Infra), Darren Liu, Dev (Devashish) Shankar, Max Leung, Nick Riasanovsky, Hao Yan, Manman Ren, Yuanwei (Kevin) Fang

Published: 2026-10-01T22:26:57Z

Content type: article

Language: en

Sources: [PyTorch](<https://devfeed.tech/sources/blog-pytorch.md>)

Topics: [performance-engineering](<https://devfeed.tech/topics/performance-engineering.md>), [CUDA Tile](<https://devfeed.tech/topics/cuda-tile.md>), [NVIDIA Blackwell](<https://devfeed.tech/topics/nvidia-blackwell.md>), [Transformer](<https://devfeed.tech/topics/transformer.md>)

Tags: [analysis](<https://devfeed.tech/tags/analysis.md>), [attention](<https://devfeed.tech/tags/attention.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [better](<https://devfeed.tech/tags/better.md>), [blackwell](<https://devfeed.tech/tags/blackwell.md>), [blog](<https://devfeed.tech/tags/blog.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [nvidia-blackwell](<https://devfeed.tech/tags/nvidia-blackwell.md>), [optimizing](<https://devfeed.tech/tags/optimizing.md>)

### AI overview

This article describes Jagged Flash Attention (JFA), the attention kernel used by Meta's Generative Ads Model, implemented with TLX on NVIDIA Blackwell. It reports that the TLX kernel uses roughly one-third as much code as the compared FlashAttention-4 kernels and outperforms that version on production jagged shapes, averaging about 13% faster forward and 50% faster backward passes. The article explains warp specialization, memory pipelining, load balancing, and other optimizations, then describes MXFP8 and block-sparse variants.

### Source excerpt

TL;DR In this blog post, we present our work on Jagged Flash Attention (JFA) -- the attention kernel behind Meta's Generative Ads Model (GEM) -- on NVIDIA Blackwell (B200), built...

## Translating CUDA Tile Operations from Python to Rust Using Agentic AI

DevFeed: [Translating CUDA Tile Operations from Python to Rust Using Agentic AI](<https://devfeed.tech/articles/translating-cuda-tile-operations-from-python-to-rust-using-agentic-ai-31486.md>)

Original publisher: [Read original article](<https://developer.nvidia.com/blog/translating-cuda-tile-operations-from-python-to-rust-using-agentic-ai/>)

Author: Tanya Lenz

Published: 2026-09-16T16:28:59Z

Content type: tutorial

Language: en

Sources: [NVIDIA Developer](<https://devfeed.tech/sources/nvidia-developer.md>), [NVIDIA Technical Blog](<https://devfeed.tech/sources/nvidia-technical-blog.md>)

Topics: [CUDA Tile](<https://devfeed.tech/topics/cuda-tile.md>), [Rust](<https://devfeed.tech/topics/rust.md>), [Agent Skills](<https://devfeed.tech/topics/agent-skills.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [AI Agent](<https://devfeed.tech/topics/ai-agent.md>), [GPU](<https://devfeed.tech/topics/gpu.md>), [Benchmark](<https://devfeed.tech/topics/benchmark.md>), [Compiler](<https://devfeed.tech/topics/compiler.md>)

Tags: [agent-skill](<https://devfeed.tech/tags/agent-skill.md>), [agentic-ai](<https://devfeed.tech/tags/agentic-ai.md>), [agentic-ai-generative-ai](<https://devfeed.tech/tags/agentic-ai-generative-ai.md>), [ai-agent](<https://devfeed.tech/tags/ai-agent.md>), [benchmarking](<https://devfeed.tech/tags/benchmarking.md>), [cuda](<https://devfeed.tech/tags/cuda.md>), [cuda-tile](<https://devfeed.tech/tags/cuda-tile.md>), [cutile](<https://devfeed.tech/tags/cutile.md>), [developer-tools-techniques](<https://devfeed.tech/tags/developer-tools-techniques.md>), [featured](<https://devfeed.tech/tags/featured.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [mixture-of-experts-moe](<https://devfeed.tech/tags/mixture-of-experts-moe.md>), [python](<https://devfeed.tech/tags/python.md>), [rust](<https://devfeed.tech/tags/rust.md>)

### AI overview

This NVIDIA developer article explains a multi-agent workflow for translating cuTile Python and Triton-TileIR GPU kernels into cuTile Rust. The team ported 24 public TileGym operators, covering about 40 kernels, and achieved 99.5% of cuTile Python performance on average, with correctness and performance checks at each stage.

### Source excerpt

cuTile Rust (cutile-rs) is a tile-based system for safe, idiomatic GPU kernel authoring in the Rust programming language. Extending the Rust ownership model to...

## Making FlashAttention-4 faster for inference

DevFeed: [Making FlashAttention-4 faster for inference](<https://devfeed.tech/articles/making-flashattention-4-faster-for-inference-79853.md>)

Original publisher: [Read original article](<https://modal.com/blog/flash-attention-4-faster>)

Author: David Wang

Published: 2026-06-11T12:00:00Z

Content type: article

Language: en

Sources: [Modal](<https://devfeed.tech/sources/modal.md>)

Topics: [Inference](<https://devfeed.tech/topics/inference.md>), [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [CUDA Tile](<https://devfeed.tech/topics/cuda-tile.md>), [TensorRT](<https://devfeed.tech/topics/tensorrt.md>), [Multi-GPU](<https://devfeed.tech/topics/multi-gpu.md>)

Tags: [architecture](<https://devfeed.tech/tags/architecture.md>), [attention](<https://devfeed.tech/tags/attention.md>), [bandwidth](<https://devfeed.tech/tags/bandwidth.md>), [cache](<https://devfeed.tech/tags/cache.md>), [fp8](<https://devfeed.tech/tags/fp8.md>), [inference](<https://devfeed.tech/tags/inference.md>)

### AI overview

Modal describes changes to FlashAttention-4 that improve large language model inference, especially decode-heavy workloads. The work includes FP8 support, arbitrary KV cache page sizes, splitting computation across the KV dimension, and handling smaller query sizes. The article reports measured throughput gains for several changes and explains their performance tradeoffs.

### Source excerpt

What part of "dtype = 'fp8', num_splits = 0, pack_gqa = True, q_stage = 1, page_size = 1" do you not understand?

## We reverse-engineered Flash Attention 4

DevFeed: [We reverse-engineered Flash Attention 4](<https://devfeed.tech/articles/we-reverse-engineered-flash-attention-4-79932.md>)

Original publisher: [Read original article](<https://modal.com/blog/reverse-engineer-flash-attention-4>)

Author: Timothy Feng

Published: 2025-09-26T00:00:00Z

Content type: article

Language: en

Sources: [Modal](<https://devfeed.tech/sources/modal.md>)

Topics: [quantization](<https://devfeed.tech/topics/quantization.md>), [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [Transformer](<https://devfeed.tech/topics/transformer.md>), [CUDA Tile](<https://devfeed.tech/topics/cuda-tile.md>), [Tensor Cores](<https://devfeed.tech/topics/tensor-cores.md>)

Tags: [ai-workloads](<https://devfeed.tech/tags/ai-workloads.md>), [algorithm](<https://devfeed.tech/tags/algorithm.md>), [allows](<https://devfeed.tech/tags/allows.md>), [async](<https://devfeed.tech/tags/async.md>), [bandwidth](<https://devfeed.tech/tags/bandwidth.md>), [benchmarking](<https://devfeed.tech/tags/benchmarking.md>), [code](<https://devfeed.tech/tags/code.md>), [flash](<https://devfeed.tech/tags/flash.md>)

### AI overview

The article reverse-engineers Flash Attention 4, a CUDA kernel for Transformer attention, and describes its asynchronous, warp-specialized pipeline on Nvidia Blackwell GPUs. It highlights approximate exponentiation and a change to softmax rescaling that reportedly reduces correction operations by a factor of 10. The write-up also discusses how programmer-managed asynchrony increases kernel complexity.

### Source excerpt

Asynchrony, fast approximate exponents, and 10x more efficient softmax.