# kernels

Hugging Face Kernels is a project and Python package for building, distributing, and loading compatible compute kernels from the Hub.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## CertiK and Columbia University Bring Math Proofs to the GPU Code Behind Every Large AI Model

DevFeed: [CertiK and Columbia University Bring Math Proofs to the GPU Code Behind Every Large AI Model](<https://devfeed.tech/articles/certik-and-columbia-university-bring-math-proofs-to-the-gpu-code-behind-every-large-ai-model-146451.md>)

Original publisher: [Read original article](<https://hackernoon.com/certik-and-columbia-university-bring-math-proofs-to-the-gpu-code-behind-every-large-ai-model?source=rss>)

Author: Ishan Pandey

Published: 2026-11-20T14:53:35Z

Content type: article

Language: en

Sources: [HackerNoon](<https://devfeed.tech/sources/hackernoon.md>)

Topics: [kernels](<https://devfeed.tech/topics/kernels.md>), [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [ai-model](<https://devfeed.tech/tags/ai-model.md>), [bugs](<https://devfeed.tech/tags/bugs.md>), [certik](<https://devfeed.tech/tags/certik.md>), [code](<https://devfeed.tech/tags/code.md>), [columbia-university](<https://devfeed.tech/tags/columbia-university.md>), [cybersecurity](<https://devfeed.tech/tags/cybersecurity.md>), [deepseek](<https://devfeed.tech/tags/deepseek.md>), [flashattention](<https://devfeed.tech/tags/flashattention.md>), [good-company](<https://devfeed.tech/tags/good-company.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [kernels](<https://devfeed.tech/tags/kernels.md>), [math](<https://devfeed.tech/tags/math.md>), [production](<https://devfeed.tech/tags/production.md>), [proofs](<https://devfeed.tech/tags/proofs.md>), [software-engineering](<https://devfeed.tech/tags/software-engineering.md>), [technology](<https://devfeed.tech/tags/technology.md>), [web3](<https://devfeed.tech/tags/web3.md>)

### AI overview

TensorLean uses machine-checked Lean proofs to verify GPU kernels at Nvidia's PTX instruction level, accounting for overlapping data transfers, changing layouts, and lower-precision arithmetic within stated tolerances. Applied to more than 100 production kernels from DeepSeek and FlashAttention, it found two bugs in DeepSeek kernels and 23 in FlashAttention; failed proofs identify breaking cases, and repaired code can be checked again. Unlike tests on selected inputs, formal proofs establish requirements across a defined input range. The article describes a workflow in which teams reverify each optimization and retain proofs alongside kernels, with reusable specifications intended to ease application across related versions. The excerpt presents this as a way to reduce risks from silent computational errors and costly reruns, while noting that adoption by labs and library maintainers remains the next test.

### Source excerpt

CertiK and Columbia's TensorLean uses machine-checked proofs to verify 100+ production AI kernels from DeepSeek and FlashAttention, finding 25 bugs.

## HiSA: Sparsifying the Hidden Dimension of RoPE Attention in LLMs

DevFeed: [HiSA: Sparsifying the Hidden Dimension of RoPE Attention in LLMs](<https://devfeed.tech/articles/hisa-sparsifying-the-hidden-dimension-of-rope-attention-in-llms-145170.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.26363>)

Author: Ambroise Borbely, Mathieu Salzmann, Olivier Verscheure

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>), [Transformer](<https://devfeed.tech/topics/transformer.md>), [kernels](<https://devfeed.tech/topics/kernels.md>)

Tags: [attention](<https://devfeed.tech/tags/attention.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [flashattention](<https://devfeed.tech/tags/flashattention.md>), [inference](<https://devfeed.tech/tags/inference.md>), [large-language-models-llms](<https://devfeed.tech/tags/large-language-models-llms.md>), [llms](<https://devfeed.tech/tags/llms.md>), [memory](<https://devfeed.tech/tags/memory.md>), [performance](<https://devfeed.tech/tags/performance.md>), [rope](<https://devfeed.tech/tags/rope.md>), [sparsity](<https://devfeed.tech/tags/sparsity.md>)

### AI overview

HiSA makes attention sparse in the hidden feature dimension: it keeps query and key projections intact but computes only part of their dot product, using a static, parameter-free rule based on token positions and RoPE structure. The authors report that models trained from scratch with HiSA outperform those using the full dot product while computing about one third as many query-key products. At inference, they report faster decoding and lower memory use, and describe FlashAttention-3 kernels for both training and inference. They also report that dense attention learns to suppress the score components HiSA omits. These findings are from the supplied abstract.

### Source excerpt

arXiv:2611.26363v1 Announce Type: new Abstract: Attention in large language models (LLMs) scores every query against every key with a full dot product. We investigate whether this dot product must be computed completely for attention to be effective. We propose Hidden Sparse Attention (HiSA), which keeps the query and key projections intact but only computes a partial dot product, following a static, parameter-free rule set by the token positions: attention becomes sparse in the hidden (feature) dimension, an axis much less explored than token sparsity, rather than over tokens as in almost all sparse attention methods. Based on the structure of rotary position embeddings (RoPE), we decompose keys and queries into parts which can be omitted when computing attention. By training LLMs from scratch with HiSA, we find that this partial dot product does not hurt but improves performance compared with the complete product while computing only about a third of the query-key products. During inference HiSA improves decoding speed and shrinks memory footprint. We also extend HiSA to FlashAttention-3 kernels for training and inference. Finally, we motivate our approach by showing that dense attention learns to suppress the attention score parts that HiSA drops. Code is available at https://github.com/ambroise-b/hisa.

## SANTA++: Sampling Attention through Representative Keys

DevFeed: [SANTA++: Sampling Attention through Representative Keys](<https://devfeed.tech/articles/santa-sampling-attention-through-representative-keys-143316.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2609.35629>)

Author: Kyle Lee, Christian Z. Pratt, Ruoyu Fang, Heekyung Lee, Avinash Lohitsa, Ryan Modafe, Kerem Y. Camsari

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Language models](<https://devfeed.tech/topics/language-models.md>), [infini-attention](<https://devfeed.tech/topics/infini-attention.md>), [kernels](<https://devfeed.tech/topics/kernels.md>), [quantization](<https://devfeed.tech/topics/quantization.md>)

Tags: [attention](<https://devfeed.tech/tags/attention.md>), [budget](<https://devfeed.tech/tags/budget.md>), [cache](<https://devfeed.tech/tags/cache.md>), [context](<https://devfeed.tech/tags/context.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [decoding](<https://devfeed.tech/tags/decoding.md>), [flash](<https://devfeed.tech/tags/flash.md>), [fp16](<https://devfeed.tech/tags/fp16.md>), [full](<https://devfeed.tech/tags/full.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [helmet](<https://devfeed.tech/tags/helmet.md>), [implementation](<https://devfeed.tech/tags/implementation.md>), [inclusion](<https://devfeed.tech/tags/inclusion.md>), [int4](<https://devfeed.tech/tags/int4.md>), [kernels](<https://devfeed.tech/tags/kernels.md>), [keys](<https://devfeed.tech/tags/keys.md>), [kv-cache](<https://devfeed.tech/tags/kv-cache.md>), [lets](<https://devfeed.tech/tags/lets.md>), [logical](<https://devfeed.tech/tags/logical.md>), [memory](<https://devfeed.tech/tags/memory.md>), [sampling](<https://devfeed.tech/tags/sampling.md>)

### AI overview

SANTA++ reduces KV-cache reads by grouping cached keys into teams, scoring one representative key per team, and sampling teams for exact within-team attention. It reweights sampled contributions by inverse inclusion probability to estimate full-cache attention; the sampling budget trades memory reads for accuracy. In the reported Qwen2.5-7B-Instruct tests at 32K context, contiguous S=1024 retained 95-98% of dense aggregate scores on LongBench v2, HELMET RAG, and RULER with 28-31% logical KV access. On RULER it beat Quest across the tested accuracy-access sweep, using 48.7% fewer reads at about 79% accuracy, and beat MagicPIG at low access. The reported Triton kernel was 1.38x faster than PyTorch Flash SDPA's attention operator in FP16; batch-eight decoding was 1.23x faster on an RTX 5090 Laptop GPU with matched INT4 weights and FP16 KV cache.

### Source excerpt

arXiv:2609.35629v2 Announce Type: replace Abstract: Attention often concentrates on a small subset of tokens in the context, but which subset matters changes from one query to the next. To exploit this changing structure, we introduce SANTA++, a training-free stochastic attention method that uses representative keys for memory-efficient selection without scanning the entire key-value (KV) cache. Cached keys are organized into teams, and the query scores one representative from each team to decide which teams to sample. We compute exact attention scores within the sampled teams and reweight each team's contribution by the inverse of its inclusion probability. This importance sampling correction estimates attention over the full cache, with a sampling budget that lets us trade memory reads for accuracy. With Qwen2.5-7B-Instruct at 32K context, contiguous SANTA++ at S=1024 retains 95-98% of dense aggregate scores on LongBench v2, HELMET RAG, and RULER using 28-31% logical KV access. On RULER, SANTA++ outperforms Quest across the tested accuracy-access sweep, using 48.7% fewer KV reads at approximately 79% accuracy. It also outperforms MagicPIG at low access. At S=1024, our Triton implementation gives a 1.38x attention-operator speedup over the Flash backend of PyTorch scaled dot-product attention (Flash SDPA) in FP16. At batch eight, full-model decoding is 1.23x faster than dense Flash SDPA with matched INT4 model weights and FP16 KV-cache precision on an RTX 5090 Laptop GPU. Our kernels are available at https://github.com/OPUSLab/santapp-kernel-demo.

## CompoundingKernel: Self-Improving from Zero to First

DevFeed: [CompoundingKernel: Self-Improving from Zero to First](<https://devfeed.tech/articles/compoundingkernel-self-improving-from-zero-to-first-137420.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.19230>)

Author: Qian Qu, Hongyang Wang, Wenbo Chen, Yichuan Li, Jingguo Liu, Xian Li, Xuan Wang, Ruochen Wen, Leo Fan, Yiming Yan, Zheshen Wang, Benoit Dumoulin

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [kernels](<https://devfeed.tech/topics/kernels.md>), [Parallel reasoning](<https://devfeed.tech/topics/parallel-reasoning.md>), [ab-mcts](<https://devfeed.tech/topics/ab-mcts.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [search](<https://devfeed.tech/tags/search.md>), [self-improving-agent](<https://devfeed.tech/tags/self-improving-agent.md>)

### AI overview

CompoundingKernel searches for GPU kernels from each problem's PyTorch definition, without reference kernels or official scores. An LLM controller selects an archived correct kernel as a parent or starts a new design, then assigns one direction to each parallel child agent; external checks determine correctness, and the archive preserves successful kernels and their lineage. On NVIDIA B200, all 235 problems had a site-verified correct kernel, and results were 4.40x faster than the reference baseline across 209 public-board problems. The system ranked first on SOL-ExecBench's public L1, L2, and Quantization boards as of 2026-11-11. At equal wall time, it beat a coded UCB-style tree on 35 of 38 problems, producing 2.4x as many correct attempts. More than 1,000 agents ran concurrently, but the authors identify reliable measurement rate--not agent count--as the scaling limit.

### Source excerpt

arXiv:2611.19230v1 Announce Type: new Abstract: We present CompoundingKernel, a self-improving agent system that writes GPU kernels by search. As of 2026-11-11, its generated kernels rank #1 on the public L1, L2 and Quantization boards of SOL-ExecBench (NVIDIA B200). They have held #1 on L1 since 2026-10-27, albeit a short loss on 2026-11-07. All 235 problems have a site-verified correct kernel; on the 209 public-board problems they are 4.40x faster than the benchmark's reference baseline. The search starts from zero: agents get each problem's PyTorch definition but no reference kernels or official scores. A per-problem archive outside the model holds every correct kernel and its lineage. An LLM controller reads it, picks a parent or a new design, and gives each parallel child agent one direction. Checks outside the model decide correctness. At equal wall time, it beat a coded UCB style tree on 35 of 38 problems by making 2.4x as many correct attempts. Over a thousand agents ran at once, and one warm restart fix raised their evaluation rate 3.6x. We find that the rate of reliable measurements, not the number of agents, limits how far such a search scales. Our code and kernels are available on URL.

## FlashRT-PAI: Fast Kernels Are Not Fast Pipelines for Physical AI

DevFeed: [FlashRT-PAI: Fast Kernels Are Not Fast Pipelines for Physical AI](<https://devfeed.tech/articles/flashrt-pai-fast-kernels-are-not-fast-pipelines-for-physical-ai-135019.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.24349>)

Author: Liang Su, David Chen, Jason Li, Kevin Guo, Teresa Lv

Published: 2026-11-16T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Physical AI](<https://devfeed.tech/topics/physical-ai.md>), [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>), [quantization](<https://devfeed.tech/topics/quantization.md>), [kernels](<https://devfeed.tech/topics/kernels.md>), [Jetson AGX Thor](<https://devfeed.tech/topics/jetson-agx-thor.md>), [lerobot](<https://devfeed.tech/topics/lerobot.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [ai-systems](<https://devfeed.tech/tags/ai-systems.md>), [co-design](<https://devfeed.tech/tags/co-design.md>), [compilers](<https://devfeed.tech/tags/compilers.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [data-movement](<https://devfeed.tech/tags/data-movement.md>), [drop-in](<https://devfeed.tech/tags/drop-in.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [faster](<https://devfeed.tech/tags/faster.md>), [fp16](<https://devfeed.tech/tags/fp16.md>), [fusion](<https://devfeed.tech/tags/fusion.md>), [hardware](<https://devfeed.tech/tags/hardware.md>), [hugging-face](<https://devfeed.tech/tags/hugging-face.md>), [inference](<https://devfeed.tech/tags/inference.md>), [inference-engine](<https://devfeed.tech/tags/inference-engine.md>), [jetson](<https://devfeed.tech/tags/jetson.md>), [jetson-agx-thor](<https://devfeed.tech/tags/jetson-agx-thor.md>), [latency](<https://devfeed.tech/tags/latency.md>), [launch](<https://devfeed.tech/tags/launch.md>), [physical-ai](<https://devfeed.tech/tags/physical-ai.md>), [pipelines](<https://devfeed.tech/tags/pipelines.md>)

### AI overview

FLASHRT-PAI targets physical AI inference latency at the pipeline level: it fuses kernels and overlaps launches, uses an execution contract to reuse state across decisions and selectively refresh stages, and provides capsules to snapshot, restore, or fork persistent state. It also offers drop-in kernel replacement for unmodified PyTorch models, gated on parity and latency. Across six policies on RTX 5090 and four on Jetson AGX Thor, it reports 1.3-3.7x speedups over official compiled paths at matched 16-bit precision; pipeline co-design combined with reduced precision yields 3.38-12.66x over official eager implementations. In LIBERO-10, four-bit pi0.5 reports 94.04% success on Thor and 93.32% on RTX 5090, versus 92.4% for the measured official FP16 reference.

### Source excerpt

arXiv:2610.24349v2 Announce Type: replace Abstract: Vision-language-action (VLA), video-action (VA), and world action models (WAMs) form the backbone of modern physical AI systems, yet their real-time single-batch inference remains a defining bottleneck. Latency is dominated not by individual kernel computation, but by system-level overhead: thousands of short kernel launches, intermediate data movement across operator boundaries, and redundant recomputation across consecutive decisions. General-purpose graph compilers cannot address these costs, as they optimize individual calls in isolation and lack visibility into cross-call state semantics and global pipeline structure. We present FLASHRT-PAI, a low-latency inference engine built around model-hardware pipeline co-design rather than isolated kernel optimization. FLASHRT-PAI combines three core mechanisms: deep kernel fusion with launch overlap eliminates inter-operator boundaries and launch gaps; an execution contract formalizes the pipeline's stage DAG, update rates, and named buffers to enable cross-decision state reuse and selective refresh; and execution-state capsules enable fast snapshot, restore, and fork of persistent pipeline state. The system also supports drop-in acceleration of unmodified PyTorch models via parity- and latency-gated kernel replacement. Evaluation covers six policies on RTX 5090 and four on Jetson AGX Thor. At matched 16-bit precision, FLASHRT-PAI is 1.3-3.7x faster than official compiled paths. Combining pipeline co-design with reduced precision delivers 3.38-12.66x speedup over official eager implementations. In LIBERO-10, four-bit pi0.5 achieves 94.04% success on Thor and 93.32% on RTX 5090, compared with 92.4% for the measured official FP16 reference. FLASHRT-PAI is open source, distributed via Hugging Face Kernel Hub, and its opt-in pi0.5 kernels have been adopted into the LeRobot framework.

## Reconstructing Fine-grained Memory Reuse Structure in GPU Kernels

DevFeed: [Reconstructing Fine-grained Memory Reuse Structure in GPU Kernels](<https://devfeed.tech/articles/reconstructing-fine-grained-memory-reuse-structure-in-gpu-kernels-131523.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.14741>)

Author: Yanbo Zhao, Yueming Hao, Zecheng Li, Mao Lin, Shuyin Jiao, Keren Zhou, Xu Liu, Jiajia Li

Published: 2026-11-13T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Distributed Computing](<https://devfeed.tech/sources/arxiv-distributed-computing.md>)

Topics: [kernels](<https://devfeed.tech/topics/kernels.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [ai-workloads](<https://devfeed.tech/tags/ai-workloads.md>), [binaries](<https://devfeed.tech/tags/binaries.md>), [block](<https://devfeed.tech/tags/block.md>), [cs-dc](<https://devfeed.tech/tags/cs-dc.md>), [cs-pf](<https://devfeed.tech/tags/cs-pf.md>), [cs-pl](<https://devfeed.tech/tags/cs-pl.md>), [data-movement](<https://devfeed.tech/tags/data-movement.md>), [execution](<https://devfeed.tech/tags/execution.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [graph](<https://devfeed.tech/tags/graph.md>), [grid](<https://devfeed.tech/tags/grid.md>), [h100](<https://devfeed.tech/tags/h100.md>), [hpc](<https://devfeed.tech/tags/hpc.md>), [instructions](<https://devfeed.tech/tags/instructions.md>), [kernels](<https://devfeed.tech/tags/kernels.md>), [memory](<https://devfeed.tech/tags/memory.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [nvidia-rtx](<https://devfeed.tech/tags/nvidia-rtx.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [optimizations](<https://devfeed.tech/tags/optimizations.md>), [reuse](<https://devfeed.tech/tags/reuse.md>), [structure](<https://devfeed.tech/tags/structure.md>)

### AI overview

The paper introduces Topological Distance and PCDepGraph to reveal which GPU memory accesses reuse data and at what execution scope (thread, warp, block, or grid). Its tool, cuVein, reconstructs this structure from optimized binaries without sampling, generally with under 10x overhead; the authors say this avoids the limits of static analysis and the production cost of cycle-level simulation. In HPC and AI workloads on NVIDIA RTX 4090 and H100, optimizations guided by PCDepGraph eliminated up to 90.18% of memory instructions and achieved up to 2.84x kernel speedup. The reported reuse-scope distributions were consistent across the two platforms.

### Source excerpt

arXiv:2611.14741v1 Announce Type: new Abstract: Optimizing GPU data movement requires knowing which instructions access the same data and across which execution scopes reuse occurs. Existing profilers can attribute memory behavior to individual instructions but do not expose the reuse relations among them, while aliasing, dynamic indexing, and runtime parameters prevent static analysis from establishing these relations in general. Cycle-level simulation can reconstruct such relations, but at costs prohibitive for production workloads. We introduce two abstractions for representing this structure: Topological Distance, which labels each dynamic memory access with the smallest execution scope (thread, warp, block, or grid) containing its most recent reuse, and PCDepGraph, which organizes these labels into a PC-level graph of the kernel's memory reuse structure. Defined over logical execution scopes rather than transient hardware state, the representation remains stable across runs and GPU targets while directly indicating the sharing scope that candidate optimization mechanisms must cover. Our tool, cuVein, reconstructs PCDepGraph directly from optimized binaries, without sampling, at generally under 10x overhead. Across HPC and AI workloads on NVIDIA RTX 4090 and H100, optimizations guided by PCDepGraph eliminate up to 90.18% of memory instructions and yield up to 2.84x kernel speedup; reuse-scope distributions remain consistent across platforms, supporting that the representation captures program structure rather than transient hardware behavior.

## Fast Attention Kernels and Operator Fusion

DevFeed: [Fast Attention Kernels and Operator Fusion](<https://devfeed.tech/articles/fast-attention-kernels-and-operator-fusion-131340.md>)

Original publisher: [Read original article](<https://machinelearningmastery.com/fast-attention-kernels-and-operator-fusion/>)

Author: Adrian Tam

Published: 2026-11-12T17:01:10Z

Content type: article

Language: en

Sources: [MachineLearningMastery.com](<https://devfeed.tech/sources/machinelearningmastery-com.md>)

Topics: [Transformer](<https://devfeed.tech/topics/transformer.md>), [CUDA Graphs](<https://devfeed.tech/topics/cuda-graphs.md>), [vllm](<https://devfeed.tech/topics/vllm.md>), [kernels](<https://devfeed.tech/topics/kernels.md>), [Compilers and Linkers](<https://devfeed.tech/topics/compilers-and-linkers.md>), [memory pool](<https://devfeed.tech/topics/memory-pool.md>), [quantization](<https://devfeed.tech/topics/quantization.md>)

Tags: [algorithm](<https://devfeed.tech/tags/algorithm.md>), [api](<https://devfeed.tech/tags/api.md>), [attention](<https://devfeed.tech/tags/attention.md>), [cache](<https://devfeed.tech/tags/cache.md>), [compilation](<https://devfeed.tech/tags/compilation.md>), [compile](<https://devfeed.tech/tags/compile.md>), [compiler](<https://devfeed.tech/tags/compiler.md>), [compiler-optimizations](<https://devfeed.tech/tags/compiler-optimizations.md>), [cuda](<https://devfeed.tech/tags/cuda.md>), [cuda-graphs](<https://devfeed.tech/tags/cuda-graphs.md>), [decode](<https://devfeed.tech/tags/decode.md>), [flashattention](<https://devfeed.tech/tags/flashattention.md>), [fusion](<https://devfeed.tech/tags/fusion.md>), [inference-from-transformer-models](<https://devfeed.tech/tags/inference-from-transformer-models.md>), [kernels](<https://devfeed.tech/tags/kernels.md>), [operator](<https://devfeed.tech/tags/operator.md>), [torch](<https://devfeed.tech/tags/torch.md>)

### AI overview

Inference can spend more time moving intermediate tensors and launching kernels than doing arithmetic. Operator fusion combines compatible operations to reduce that traffic and launch count, though oversized or poorly shaped fused kernels can be slower, so profile the full workload. FlashAttention-style kernels tile exact attention and keep softmax statistics and outputs on chip, avoiding storage of the full quadratic score matrix; this is especially useful during long prefill. Decode often becomes more constrained by KV-cache reads. Paged attention maps logical token positions through a block table to fixed-size cache blocks, reducing fragmentation and reserved capacity without reducing the logical KV data. In vLLM, selecting a cache block size does not select the compute backend; the requested backend is validated against model and hardware. `torch.compile()` with TorchInductor fuses compatible graph regions, while CUDA Graph replay separately reduces CPU and launch overhead for eligible, repeated workloads. Graph breaks, changing shapes, numerical differences, and extra retained workspace can limit these benefits. Check profiler traces, numerical tolerances, and end-to-end speed

### Source excerpt

Overview This post is divided into four parts; they are: - Operator Fusion - FlashAttention-Style Kernels - Paged Attention Kernels - Fusion and CUDA Graphs with torch.

## Fast by Default: Compile-Time Heuristics for Helion Kernels

DevFeed: [Fast by Default: Compile-Time Heuristics for Helion Kernels](<https://devfeed.tech/articles/fast-by-default-compile-time-heuristics-for-helion-kernels-131024.md>)

Original publisher: [Read original article](<https://pytorch.org/blog/fast-by-default-compile-time-heuristics-for-helion-kernels/>)

Author: Caleb Min Sup Kim, Ethan Che, Jongsok Choi, Jason Ansel

Published: 2026-11-12T16:45:26Z

Content type: article

Language: en

Sources: [PyTorch](<https://devfeed.tech/sources/blog-pytorch.md>)

Topics: [kernels](<https://devfeed.tech/topics/kernels.md>), [CUDA Tile](<https://devfeed.tech/topics/cuda-tile.md>), [AWS Inferentia](<https://devfeed.tech/topics/aws-inferentia.md>)

Tags: [analysis](<https://devfeed.tech/tags/analysis.md>), [analyze](<https://devfeed.tech/tags/analyze.md>), [aot](<https://devfeed.tech/tags/aot.md>), [arithmetic](<https://devfeed.tech/tags/arithmetic.md>), [assembler](<https://devfeed.tech/tags/assembler.md>), [attention](<https://devfeed.tech/tags/attention.md>), [axis](<https://devfeed.tech/tags/axis.md>), [backend](<https://devfeed.tech/tags/backend.md>), [backends](<https://devfeed.tech/tags/backends.md>), [batch](<https://devfeed.tech/tags/batch.md>), [benchmarking](<https://devfeed.tech/tags/benchmarking.md>), [blackwell](<https://devfeed.tech/tags/blackwell.md>), [block](<https://devfeed.tech/tags/block.md>), [blog](<https://devfeed.tech/tags/blog.md>), [dataflow](<https://devfeed.tech/tags/dataflow.md>), [helion](<https://devfeed.tech/tags/helion.md>), [ir](<https://devfeed.tech/tags/ir.md>), [kernel](<https://devfeed.tech/tags/kernel.md>), [kernels](<https://devfeed.tech/tags/kernels.md>), [pytorch](<https://devfeed.tech/tags/pytorch.md>), [reduction](<https://devfeed.tech/tags/reduction.md>)

### AI overview

Helion's compile-time heuristics analyze kernel IR and dataflow to choose settings such as block sizes, warps, and pipeline stages without benchmarking candidates. They classify kernels as matmul, reduction, or pointwise and estimate register, shared-memory, and tensor-memory use to guide choices; unknown compile-time extents still fall back to Helion's prior default. On the reported NVIDIA benchmarks, the heuristic improved results over that default: 1.18x handwritten Flash Linear Attention performance, 1.27x vLLM CUDA performance for reduction kernels, and 1.09x torch.compile max-autotune on a reduction suite. It also provided strong initial configs for autotuning, though seeded and unseeded searches reached nearly equal final results. The estimates are approximate, results include a jagged HSTU attention regression, and the heuristics are enabled for H100 and B200 on Triton; other backends remain unevaluated.

### Source excerpt

TL;DR Helion now uses compile-time heuristics that analyze a kernel's IR and dataflow to pick fast settings without slow autotuning. It classifies the kernel as matmul, reduction, or pointwise and... The post Fast by Default: Compile-Time Heuristics for Helion Kernels appeared first on PyTorch.

## KOSMOS: Self-scheduling Persistent Kernels for Communication-Compute Overlap

DevFeed: [KOSMOS: Self-scheduling Persistent Kernels for Communication-Compute Overlap](<https://devfeed.tech/articles/kosmos-self-scheduling-persistent-kernels-for-communication-compute-overlap-120197.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.07477>)

Author: Alexander Magro, Aristotle Martin, Ye Wang, Wen Chen

Published: 2026-11-06T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Distributed Computing](<https://devfeed.tech/sources/arxiv-distributed-computing.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [kernels](<https://devfeed.tech/topics/kernels.md>), [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>), [scheduling](<https://devfeed.tech/topics/scheduling.md>), [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>)

Tags: [amd](<https://devfeed.tech/tags/amd.md>), [amd-instinct-mi355x](<https://devfeed.tech/tags/amd-instinct-mi355x.md>), [collective](<https://devfeed.tech/tags/collective.md>), [communication](<https://devfeed.tech/tags/communication.md>), [compute](<https://devfeed.tech/tags/compute.md>), [consumer](<https://devfeed.tech/tags/consumer.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-dc](<https://devfeed.tech/tags/cs-dc.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-pf](<https://devfeed.tech/tags/cs-pf.md>), [data](<https://devfeed.tech/tags/data.md>), [data-readiness](<https://devfeed.tech/tags/data-readiness.md>), [deepep](<https://devfeed.tech/tags/deepep.md>), [deepseek](<https://devfeed.tech/tags/deepseek.md>), [deepseek-v3](<https://devfeed.tech/tags/deepseek-v3.md>), [demand](<https://devfeed.tech/tags/demand.md>), [expert-parallelism](<https://devfeed.tech/tags/expert-parallelism.md>), [gpus](<https://devfeed.tech/tags/gpus.md>), [instinct](<https://devfeed.tech/tags/instinct.md>), [kernels](<https://devfeed.tech/tags/kernels.md>), [launch](<https://devfeed.tech/tags/launch.md>), [layer](<https://devfeed.tech/tags/layer.md>), [overlap](<https://devfeed.tech/tags/overlap.md>), [scheduling](<https://devfeed.tech/tags/scheduling.md>)

### AI overview

KOSMOS overlaps communication and computation in transformer parallelism by using persistent kernel workgroups that switch roles as data become ready, following queues ordered by consumer demand. This avoids a scheduler, per-call cost model, or tuned communication-computation split. It fuses each TP+SP collective with its GEMM and runs an entire expert layer, from router to fold, in one launch per direction. On AMD Instinct MI350X and MI355X, it reports 1.10-1.28x faster TP+SP end-to-end Megatron-LM training in BF16 and 1.10-1.25x in MXFP8. For DeepSeek-V3 and Qwen3-235B proxies, it reports 1.29-1.83x faster end-to-end training than DeepEP.

### Source excerpt

arXiv:2611.07477v1 Announce Type: cross Abstract: Tensor and sequence parallelism (TP+SP) and expert parallelism (EP) put collectives on the critical path of transformer layers. State-of-the-art fused kernels mitigate this with fixed roles or schedules, but this rigidity leaves large idle windows. Predicting a communication-computation split does not remove them either; a predetermined split cannot follow data readiness. We present KOSMOS, a library for self-scheduling persistent kernels whose workgroups use queues ordered by consumer demand to switch between communication and computation roles as data become ready, with no scheduler, per-call cost model, or tuned split. KOSMOS fuses every TP+SP collective of a transformer layer with its GEMM, and runs an entire expert layer, from router to fold, in one launch per direction. Evaluated on AMD Instinct MI350X and AMD Instinct MI355X GPUs, KOSMOS speeds up TP+SP end-to-end training in Megatron-LM by 1.10-1.28x in BF16 and 1.10-1.25x in MXFP8. KOSMOS accelerates end-to-end training of DeepSeek-V3 and Qwen3-235B proxies by 1.29-1.83x over DeepEP.

## Turning Conv2D Into A Practical GPU Compute Pattern

DevFeed: [Turning Conv2D Into A Practical GPU Compute Pattern](<https://devfeed.tech/articles/turning-conv2d-into-a-practical-gpu-compute-pattern-118755.md>)

Original publisher: [Read original article](<https://semiengineering.com/turning-conv2d-into-a-practical-gpu-compute-pattern/>)

Author: Mario Florea

Published: 2026-11-05T08:05:35Z

Content type: article

Language: en

Sources: [Semiconductor Engineering](<https://devfeed.tech/sources/semiconductor-engineering.md>)

Topics: [kernels](<https://devfeed.tech/topics/kernels.md>), [High-Performance Computing](<https://devfeed.tech/topics/high-performance-computing.md>), [vectorization techniques](<https://devfeed.tech/topics/vectorization-techniques.md>)

Tags: [acceleration](<https://devfeed.tech/tags/acceleration.md>), [ai](<https://devfeed.tech/tags/ai.md>), [arithmetic](<https://devfeed.tech/tags/arithmetic.md>), [article](<https://devfeed.tech/tags/article.md>), [bandwidth](<https://devfeed.tech/tags/bandwidth.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [blogs-security](<https://devfeed.tech/tags/blogs-security.md>), [boundary](<https://devfeed.tech/tags/boundary.md>), [branch](<https://devfeed.tech/tags/branch.md>), [buffer](<https://devfeed.tech/tags/buffer.md>), [cache](<https://devfeed.tech/tags/cache.md>), [challenge](<https://devfeed.tech/tags/challenge.md>), [cnns](<https://devfeed.tech/tags/cnns.md>), [complexity](<https://devfeed.tech/tags/complexity.md>), [convolution](<https://devfeed.tech/tags/convolution.md>), [convolutional-neural-networks](<https://devfeed.tech/tags/convolutional-neural-networks.md>), [critical](<https://devfeed.tech/tags/critical.md>), [data-movement](<https://devfeed.tech/tags/data-movement.md>), [deep-learning](<https://devfeed.tech/tags/deep-learning.md>), [efficiency](<https://devfeed.tech/tags/efficiency.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [gpu-compute](<https://devfeed.tech/tags/gpu-compute.md>), [gpus](<https://devfeed.tech/tags/gpus.md>), [imagination-technologies](<https://devfeed.tech/tags/imagination-technologies.md>), [iot-and-security](<https://devfeed.tech/tags/iot-and-security.md>), [kernels](<https://devfeed.tech/tags/kernels.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [ml](<https://devfeed.tech/tags/ml.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [sharpening-your-edge](<https://devfeed.tech/tags/sharpening-your-edge.md>)

### AI overview

The excerpt presents Conv2D as structured matrix multiplication with accumulation across filter positions. With NHWC input, a 1x1 convolution maps each C_in feature vector to C_out directly, avoiding a transposition. Larger filters require repeated matrix-like contributions, so loop order matters: processing filter positions separately may refetch input through L1 or L2, while reordering can improve locality and cache reuse. The implementation also must handle padding, stride and dilation. Boundary checks are straightforward but can add cost in a hot loop; redirecting invalid reads to a zero-filled buffer moves that work into setup and may be worth testing. The guidance is to address data reuse and memory traffic before low-level instruction tuning, while recognizing that performance depends on execution and layout as well as the mathematics. The supplied text is an excerpt.

### Source excerpt

A reusable optimization mindset for engineers building GPU compute kernels. The post Turning Conv2D Into A Practical GPU Compute Pattern appeared first on Semiconductor Engineering.

## A Closer Look at Phonon

DevFeed: [A Closer Look at Phonon](<https://devfeed.tech/articles/a-closer-look-at-phonon-121146.md>)

Original publisher: [Read original article](<https://gradium.ai/blog/phonon-closer-look>)

Author: Gradium; Hugging Face

Published: 2026-11-05T00:00:00Z

Content type: article

Language: en

Sources: [Gradium](<https://devfeed.tech/sources/gradium.md>)

Topics: [Open Source](<https://devfeed.tech/topics/open-source.md>), [hugging face](<https://devfeed.tech/topics/hugging-face.md>), [spaces](<https://devfeed.tech/topics/spaces.md>), [huggingface\_hub](<https://devfeed.tech/topics/huggingface-hub.md>), [Tool](<https://devfeed.tech/topics/tool.md>), [kernels](<https://devfeed.tech/topics/kernels.md>), [open-llm-leaderboard](<https://devfeed.tech/topics/open-llm-leaderboard.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [announcement](<https://devfeed.tech/tags/announcement.md>), [apache-2-0](<https://devfeed.tech/tags/apache-2-0.md>), [apache-2-0-tts](<https://devfeed.tech/tags/apache-2-0-tts.md>), [apple](<https://devfeed.tech/tags/apple.md>), [apple-watch](<https://devfeed.tech/tags/apple-watch.md>), [apps](<https://devfeed.tech/tags/apps.md>), [arabic](<https://devfeed.tech/tags/arabic.md>), [arm](<https://devfeed.tech/tags/arm.md>), [audio](<https://devfeed.tech/tags/audio.md>), [autonomous](<https://devfeed.tech/tags/autonomous.md>), [backends](<https://devfeed.tech/tags/backends.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [browser](<https://devfeed.tech/tags/browser.md>), [chat-template](<https://devfeed.tech/tags/chat-template.md>), [checkpoint](<https://devfeed.tech/tags/checkpoint.md>), [cloud](<https://devfeed.tech/tags/cloud.md>), [codec](<https://devfeed.tech/tags/codec.md>), [hugging-face](<https://devfeed.tech/tags/hugging-face.md>), [multilingual-tts](<https://devfeed.tech/tags/multilingual-tts.md>), [on-device-tts](<https://devfeed.tech/tags/on-device-tts.md>), [open-tts-leaderboard](<https://devfeed.tech/tags/open-tts-leaderboard.md>), [open-weight-tts](<https://devfeed.tech/tags/open-weight-tts.md>), [phonon](<https://devfeed.tech/tags/phonon.md>), [phonon-31m](<https://devfeed.tech/tags/phonon-31m.md>), [phonon-40m](<https://devfeed.tech/tags/phonon-40m.md>), [pocket-tts](<https://devfeed.tech/tags/pocket-tts.md>), [ptts](<https://devfeed.tech/tags/ptts.md>), [time-to-first-audio](<https://devfeed.tech/tags/time-to-first-audio.md>), [transformers-tts](<https://devfeed.tech/tags/transformers-tts.md>), [tts](<https://devfeed.tech/tags/tts.md>)

### AI overview

Phonon is an open-weights text-to-speech model designed to run locally without a server or network, keeping typed text on the device. The release includes models of about 65 MB and 75 MB, more than 100 built-in voices, and support across English, French, German, Spanish, and Portuguese. Phonon 31m is Apache 2.0, including commercial use; Phonon 40m is non-commercial. The authors attribute the models' compact quality to flow matching and report that Phonon ranks first in German and Spanish intelligibility and in the top three in each evaluated language, while leading CPU streaming time-to-first-audio benchmarks. Their ptts framework targets single-stream inference across CPU, GPU, and NPU backends; the Transformers implementation targets GPU throughput with batched generation and full-graph torch.compile. Custom voices can be cloned from a short recording with the speaker's consent or designed in text. The leaderboard is described as new, and the reported rankings apply to the languages evaluated.

### Source excerpt

Two weeks after opening up Phonon, the story of the launch: why we built it, what it can do, the demos that came with it, and how it did on Hugging Face's Open TTS Leaderboard.

## AMD Releases ZenDNN 6.1 With New Kernels & Optimizations

DevFeed: [AMD Releases ZenDNN 6.1 With New Kernels & Optimizations](<https://devfeed.tech/articles/amd-releases-zendnn-6-1-with-new-kernels-optimizations-116096.md>)

Original publisher: [Read original article](<https://www.phoronix.com/news/AMD-ZenDNN-6.1>)

Author: Michael Larabel

Published: 2026-11-04T10:19:15Z

Content type: release

Language: en

Sources: [Phoronix](<https://devfeed.tech/sources/phoronix.md>)

Topics: [releases](<https://devfeed.tech/topics/releases.md>), [kernels](<https://devfeed.tech/topics/kernels.md>)

Tags: [amd](<https://devfeed.tech/tags/amd.md>), [api](<https://devfeed.tech/tags/api.md>), [custom](<https://devfeed.tech/tags/custom.md>), [deep-learning](<https://devfeed.tech/tags/deep-learning.md>), [desktop-linux](<https://devfeed.tech/tags/desktop-linux.md>), [faster](<https://devfeed.tech/tags/faster.md>), [inference](<https://devfeed.tech/tags/inference.md>), [int8](<https://devfeed.tech/tags/int8.md>), [kernels](<https://devfeed.tech/tags/kernels.md>), [linux-benchmarking](<https://devfeed.tech/tags/linux-benchmarking.md>), [linux-hardware-benchmarks](<https://devfeed.tech/tags/linux-hardware-benchmarks.md>), [linux-hardware-reviews](<https://devfeed.tech/tags/linux-hardware-reviews.md>), [linux-how-to](<https://devfeed.tech/tags/linux-how-to.md>), [linux-performance](<https://devfeed.tech/tags/linux-performance.md>), [linux-server-benchmarks](<https://devfeed.tech/tags/linux-server-benchmarks.md>), [open-source-graphics](<https://devfeed.tech/tags/open-source-graphics.md>), [phoronix](<https://devfeed.tech/tags/phoronix.md>), [phoronix-test-suite](<https://devfeed.tech/tags/phoronix-test-suite.md>), [software](<https://devfeed.tech/tags/software.md>), [ubuntu-benchmarks](<https://devfeed.tech/tags/ubuntu-benchmarks.md>), [ubuntu-hardware](<https://devfeed.tech/tags/ubuntu-hardware.md>)

### AI overview

AMD ZenDNN 6.1 extends its LowOHA inference stack with faster group MATMUL custom kernels, FP16 and DA8W4 kernels, and MATMUL custom kernels for DA8W4, DA8W8, INT8, and BF16. It also adds symmetric INT8 prepacking and other improvements. The excerpt does not quantify performance gains or detail requirements. ZenDNN is described as oneDNN API compatible; AMD is continuing development in ZenDNN alongside the initial AMD Zen optimizations in oneDNN 3.14.

### Source excerpt

Last month Intel and AMD jointly announced they would be collaborating on software optimizations with PyTorch/oneDNN. That already led to the UXL Foundation's release of oneDNN 3.14 with initial AMD Zen optimizations. It looks though like for now at least AMD will also continue to concurrently push new work into their ZenDNN library and not exclusively tied to oneDNN. Out today is ZenDNN 6.1...

## FlyWheel: Attention Kernels Built for the TPU

DevFeed: [FlyWheel: Attention Kernels Built for the TPU](<https://devfeed.tech/articles/flywheel-attention-kernels-built-for-the-tpu-114544.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.01426>)

Author: Zishuo Bao, Zeshen Zhang, Linkun Chen, Yucheng Lu

Published: 2026-11-03T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [kernels](<https://devfeed.tech/topics/kernels.md>), [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>), [vllm](<https://devfeed.tech/topics/vllm.md>), [performance-optimization](<https://devfeed.tech/topics/performance-optimization.md>)

Tags: [attention](<https://devfeed.tech/tags/attention.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [kernels](<https://devfeed.tech/tags/kernels.md>), [latency](<https://devfeed.tech/tags/latency.md>), [linear](<https://devfeed.tech/tags/linear.md>), [prefill](<https://devfeed.tech/tags/prefill.md>), [qwen3-8-27b](<https://devfeed.tech/tags/qwen3-8-27b.md>), [reuse](<https://devfeed.tech/tags/reuse.md>), [serving](<https://devfeed.tech/tags/serving.md>), [tensor-processing-units](<https://devfeed.tech/tags/tensor-processing-units.md>), [throughput](<https://devfeed.tech/tags/throughput.md>), [tpu](<https://devfeed.tech/tags/tpu.md>), [vllm](<https://devfeed.tech/tags/vllm.md>)

### AI overview

FlyWheel targets TPU attention kernels' underuse of matrix units, caused by repeated operand loads, vector-heavy work and idle time from dependencies. It reduces vector work by reformulating softmax and triangular inversion, coordinates layouts to reuse matrix operands, and overlaps independent computation to hide latency. On TPU v6e and v7x, it reports up to 4.14x faster softmax attention than RPA and 1.80x faster than Splash Attention, reaching 81% of peak BF16 throughput on v6e; Gated DeltaNet prefill improves by up to 3.61x. In vLLM serving Qwen3.8-27B, prefill throughput rises by up to 1.56x with comparable accuracy. These are reported maxima; the excerpt does not specify benchmark conditions.

### Source excerpt

arXiv:2611.01426v1 Announce Type: cross Abstract: Attention kernels on Tensor Processing Units (TPUs) often underutilize their matrix units (MXUs) because of repeated matrix operand loads, vector-intensive computation, and dependencies that leave MXUs idle. We present FlyWheel, a set of fast TPU kernels for both softmax and linear attention. FlyWheel reformulates softmax and triangular inversion to reduce vector work, while coordinating data layouts to improve matrix operand reuse. It then overlaps independent computation to hide execution latency. We apply these techniques to both softmax attention and Gated DeltaNet on TPU v6e and v7x. FlyWheel speeds up softmax attention by up to $4.14\times$ over the state-of-the-art RPA kernel and $1.80\times$ over Splash Attention, reaching 81% of peak BF16 throughput on v6e. It also speeds up Gated DeltaNet prefill by up to $3.61\times$. Integrated into vLLM serving Qwen3.8-27B, FlyWheel improves prefill throughput by up to $1.56\times$ while maintaining comparable accuracy.

## 📉 98% GPU Utilization and a 6x Speedup Left on the Table

DevFeed: [📉 98% GPU Utilization and a 6x Speedup Left on the Table](<https://devfeed.tech/articles/98-gpu-utilization-and-a-6x-speedup-left-on-the-table-91251.md>)

Original publisher: [Read original article](<https://read.bytesizeddesign.com/p/98-percent-gpu-utilization-6x-speedup>)

Author: Byte-Sized Design

Published: 2026-10-20T16:16:53Z

Content type: article

Language: en

Sources: [Byte-Sized Design](<https://devfeed.tech/sources/byte-sized-design.md>)

Topics: [gpu monitoring](<https://devfeed.tech/topics/gpu-monitoring.md>), [qwen](<https://devfeed.tech/topics/qwen.md>), [kernels](<https://devfeed.tech/topics/kernels.md>), [pip](<https://devfeed.tech/topics/pip.md>)

Tags: [activation](<https://devfeed.tech/tags/activation.md>), [agentic](<https://devfeed.tech/tags/agentic.md>), [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-agents](<https://devfeed.tech/tags/ai-agents.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [attention](<https://devfeed.tech/tags/attention.md>), [batch](<https://devfeed.tech/tags/batch.md>), [bias](<https://devfeed.tech/tags/bias.md>), [bug](<https://devfeed.tech/tags/bug.md>), [chat](<https://devfeed.tech/tags/chat.md>), [community](<https://devfeed.tech/tags/community.md>), [compute](<https://devfeed.tech/tags/compute.md>), [container](<https://devfeed.tech/tags/container.md>), [cpu](<https://devfeed.tech/tags/cpu.md>), [databricks](<https://devfeed.tech/tags/databricks.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [gpu-utilization](<https://devfeed.tech/tags/gpu-utilization.md>), [kernels](<https://devfeed.tech/tags/kernels.md>), [qwen](<https://devfeed.tech/tags/qwen.md>)

### AI overview

A Qwen 27B fine-tune took 7h44m despite 98% GPU utilization; installing the model's supported `flash-linear-attention` and `causal-conv1d` kernels reduced runtime to 2h05m, then packing examples instead of padding them cut it to 1h17m. A PyTorch trace revealed 269,448 GPU launches per step and CPU dispatch slower than GPU execution; after kernel installation, launches fell 15-fold. Packing uses boundary-reset `position_ids` to preserve attention between examples and reduced batches by about 36% on this dataset, though it makes the input pipeline more complex and mask bugs could silently harm quality. The article recommends tracing a step and tracking throughput alongside utilization, since utilization stayed at 98% throughout. It does not report final tokens per second, GPU count, or CPU/GPU timing after the kernel fix.

### Source excerpt

A Qwen 27B fine-tune showed 98% GPU utilization and ran 6x slower than it should. Two fixes from Databricks: install the fused kernels, pack the batches.

## PwnEscape (CVE-2026-99999): Patched kernels available in testing

DevFeed: [PwnEscape (CVE-2026-99999): Patched kernels available in testing](<https://devfeed.tech/articles/pwnescape-cve-2026-99999-patched-kernels-available-in-testing-88722.md>)

Original publisher: [Read original article](<https://almalinux.org/blog/2026-10-16-pwnescape-cve-2026-99999/>)

Author: Jonathan Wright Infrastructure SIG Lead; ALESCo Member

Published: 2026-10-16T20:18:35Z

Content type: news

Language: en

Sources: [AlmaLinux](<https://devfeed.tech/sources/blog-on-almalinux.md>)

Topics: [cve remediation](<https://devfeed.tech/topics/cve-remediation.md>), [Kernel](<https://devfeed.tech/topics/kernel.md>), [virtual machines](<https://devfeed.tech/topics/virtual-machines.md>), [kernels](<https://devfeed.tech/topics/kernels.md>)

Tags: [cve](<https://devfeed.tech/tags/cve.md>), [kernels](<https://devfeed.tech/tags/kernels.md>), [linux](<https://devfeed.tech/tags/linux.md>), [linux-kernel](<https://devfeed.tech/tags/linux-kernel.md>), [vulnerability](<https://devfeed.tech/tags/vulnerability.md>)

### AI overview

The article reports PwnEscape (CVE-2026-99999), a Linux kernel flaw that can let a guest VM leak host kernel memory, crash the host, or run code as root. AlmaLinux 8, 9, and 10 are affected; patched kernels are available in the testing repository for verification before production release.

### Source excerpt

A new Linux kernel vulnerability, nicknamed PwnEscape (CVE-2026-99999), has been disclosed. It is a guest-to-host virtual machine (VM) escape: anyone who controls a guest VM can send specially crafted network packets that leak the host's kernel memory, crash the host, or, as the researcher demonstrated in their own lab, run code as root on the host. Both Red Hat and the Linux kernel CVE team rate it Critical. AlmaLinux 8, 9, and 10 are affected. Patched kernels are available in the testing repository today, and we would like your help verifying them before we release them to production.

## 2026 Retrocomputer Challenge: Vectron65 Plus Puts the Emphasis on Plus

DevFeed: [2026 Retrocomputer Challenge: Vectron65 Plus Puts the Emphasis on Plus](<https://devfeed.tech/articles/2026-retrocomputer-challenge-vectron65-plus-puts-the-emphasis-on-plus-64481.md>)

Original publisher: [Read original article](<https://hackaday.com/2026/10/03/2026-retrocomputer-challenge-vectron65-plus-puts-the-emphasis-on-plus/>)

Author: Tyler August

Published: 2026-10-03T20:00:27Z

Content type: news

Language: en

Sources: [Hackaday](<https://devfeed.tech/sources/hackaday.md>)

Topics: [Python](<https://devfeed.tech/topics/python.md>), [kernels](<https://devfeed.tech/topics/kernels.md>)

Tags: [2026-retrocomputing-challenge](<https://devfeed.tech/tags/2026-retrocomputing-challenge.md>), [6502](<https://devfeed.tech/tags/6502.md>), [commodore](<https://devfeed.tech/tags/commodore.md>), [computer](<https://devfeed.tech/tags/computer.md>), [contests](<https://devfeed.tech/tags/contests.md>), [cpu](<https://devfeed.tech/tags/cpu.md>), [gpio](<https://devfeed.tech/tags/gpio.md>), [graphics-card](<https://devfeed.tech/tags/graphics-card.md>), [homebrew](<https://devfeed.tech/tags/homebrew.md>), [homebrew-computer](<https://devfeed.tech/tags/homebrew-computer.md>), [retro](<https://devfeed.tech/tags/retro.md>), [retrocomputer](<https://devfeed.tech/tags/retrocomputer.md>), [retrocomputing](<https://devfeed.tech/tags/retrocomputing.md>), [vectron](<https://devfeed.tech/tags/vectron.md>)

### AI overview

The Vectron65 Plus is a 6502-based homebrew computer project with expansion cards, including VGA graphics and a text mode card that uses a second 6502 to handle screen output. The article describes its move from breadboard to PCB and notes upgrades such as more RAM and GPIO ports, as well as its place in the 2026 Retrocomputing contest.

### Source excerpt

[Nick Bild] evidently has a soft spot for the 6502, given that his Vectron65 Plus home computer project uses at least one of them, depending on the add-ons. You see, ...read more

## Seven stable kernels for Saturday

DevFeed: [Seven stable kernels for Saturday](<https://devfeed.tech/articles/seven-stable-kernels-for-saturday-64342.md>)

Original publisher: [Read original article](<https://lwn.net/Articles/1098404/>)

Author: jzb

Published: 2026-10-03T11:21:42Z

Content type: news

Language: en

Sources: [LWN.net](<https://devfeed.tech/sources/lwn-net.md>)

Topics: [kernels](<https://devfeed.tech/topics/kernels.md>)

Tags: [kernels](<https://devfeed.tech/tags/kernels.md>), [release](<https://devfeed.tech/tags/release.md>), [stable](<https://devfeed.tech/tags/stable.md>), [upgrade](<https://devfeed.tech/tags/upgrade.md>)

### AI overview

Greg Kroah-Hartman announced seven stable Linux kernel releases, each containing fixes throughout the tree, and advised users to upgrade.

### Source excerpt

Greg Kroah-Hartman has announced the release of the 7.2.9, 6.18.55, 6.12.112, 6.6.158, 6.1.189, 5.15.222, and 5.10.271 stable kernels. Each contains a large number of important fixes throughout the tree; users are advised to upgrade.

## Building a GPU in 24 hours and winning HackMIT

DevFeed: [Building a GPU in 24 hours and winning HackMIT](<https://devfeed.tech/articles/building-a-gpu-in-24-hours-and-winning-hackmit-61184.md>)

Original publisher: [Read original article](<https://blog.adafruit.com/2026/09/28/building-a-gpu-in-24-hours-and-winning-hackmit/>)

Author: Anne Barela

Published: 2026-09-28T15:16:18Z

Content type: article

Language: en

Sources: [Adafruit Industries](<https://devfeed.tech/sources/adafruit-industries-makers-hackers-artists-designers-and-engineers.md>)

Topics: [kernels](<https://devfeed.tech/topics/kernels.md>), [vectorization techniques](<https://devfeed.tech/topics/vectorization-techniques.md>), [Multi-GPU](<https://devfeed.tech/topics/multi-gpu.md>)

Tags: [contest](<https://devfeed.tech/tags/contest.md>), [fpga](<https://devfeed.tech/tags/fpga.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [graphics](<https://devfeed.tech/tags/graphics.md>), [mit](<https://devfeed.tech/tags/mit.md>), [multithreading](<https://devfeed.tech/tags/multithreading.md>), [rtl](<https://devfeed.tech/tags/rtl.md>), [simd](<https://devfeed.tech/tags/simd.md>)

### AI overview

A four-person team built an FPGA-based GPU in 24 hours at HackMIT and placed second. The design combined a pipelined RV32 core, 16-lane SIMD, hardware multithreading, and custom kernels. To address serialized-memory stalls, the team added an asynchronous load-store unit. Demos included Mandelbrot, a physics simulation, an LLM, and DOOM.

### Source excerpt

A team of four built a working GPU during the 24 hours of HackMIT and took second place with it. The write-up on armaangomes.com covers the architecture and the all-night scramble to get it running on an FPGA. The plan had four stages: start with a pipelined RV32 core, vectorize it into a 16-lane SIMD [...]

## Two stable kernels for Friday

DevFeed: [Two stable kernels for Friday](<https://devfeed.tech/articles/two-stable-kernels-for-friday-60183.md>)

Original publisher: [Read original article](<https://lwn.net/Articles/1096818/>)

Author: jzb

Published: 2026-09-25T15:13:25Z

Content type: release

Language: en

Sources: [LWN.net](<https://devfeed.tech/sources/lwn-net.md>)

Topics: [kernels](<https://devfeed.tech/topics/kernels.md>), [upgrade](<https://devfeed.tech/topics/upgrade.md>)

Tags: [kernels](<https://devfeed.tech/tags/kernels.md>), [release](<https://devfeed.tech/tags/release.md>), [stable](<https://devfeed.tech/tags/stable.md>), [upgrade](<https://devfeed.tech/tags/upgrade.md>)

### AI overview

A release announcement covers stable kernel versions 7.2.8 and 6.18.54, which include fixes across the kernel codebase. Users are advised to upgrade.

### Source excerpt

Greg Kroah-Hartman has announced the release of the 7.2.8 and 6.18.54 stable kernels. Each contains a number of important fixes throughout the tree; users are advised to upgrade.

## Why 2-Bit LLM Quantization Fails at the Hardware Boundary

DevFeed: [Why 2-Bit LLM Quantization Fails at the Hardware Boundary](<https://devfeed.tech/articles/why-2-bit-llm-quantization-fails-at-the-hardware-boundary-56849.md>)

Original publisher: [Read original article](<https://hackernoon.com/why-2-bit-llm-quantization-fails-at-the-hardware-boundary?source=rss>)

Author: Vivek Kumar

Published: 2026-09-21T14:50:14Z

Content type: article

Language: en

Sources: [HackerNoon](<https://devfeed.tech/sources/hackernoon.md>)

Topics: [Large Language Model](<https://devfeed.tech/topics/llm.md>), [quantization](<https://devfeed.tech/topics/quantization.md>), [Compression](<https://devfeed.tech/topics/compression.md>), [Hardware](<https://devfeed.tech/topics/hardware.md>), [Post-training optimization](<https://devfeed.tech/topics/post-training-optimization.md>), [Compiler](<https://devfeed.tech/topics/compiler.md>), [kernels](<https://devfeed.tech/topics/kernels.md>)

Tags: [2-bit-llm-quantization](<https://devfeed.tech/tags/2-bit-llm-quantization.md>), [ai-inference](<https://devfeed.tech/tags/ai-inference.md>), [ai-infrastructure](<https://devfeed.tech/tags/ai-infrastructure.md>), [compiler](<https://devfeed.tech/tags/compiler.md>), [compression](<https://devfeed.tech/tags/compression.md>), [edge-ai-optimization](<https://devfeed.tech/tags/edge-ai-optimization.md>), [floating-point](<https://devfeed.tech/tags/floating-point.md>), [hackernoon-top-story](<https://devfeed.tech/tags/hackernoon-top-story.md>), [hardware](<https://devfeed.tech/tags/hardware.md>), [hardware-acceleration](<https://devfeed.tech/tags/hardware-acceleration.md>), [kernels](<https://devfeed.tech/tags/kernels.md>), [llm](<https://devfeed.tech/tags/llm.md>), [llm-quantization](<https://devfeed.tech/tags/llm-quantization.md>), [model-compression](<https://devfeed.tech/tags/model-compression.md>), [post-training](<https://devfeed.tech/tags/post-training.md>), [quantization](<https://devfeed.tech/tags/quantization.md>)

### AI overview

This article explains why 2-bit LLM quantization can fail when deployed on custom edge accelerators. It argues that extreme compression requires coordinated design across representation, training, compilers, hardware kernels, and device runtimes because low-bit numerical differences can alter model behavior in production.

### Source excerpt

The jump to 2-bit LLM quantization isn't just about smaller integers. Explore the hardware-software boundary and why extreme compression requires co-design

## Three new stable kernels

DevFeed: [Three new stable kernels](<https://devfeed.tech/articles/three-new-stable-kernels-57615.md>)

Original publisher: [Read original article](<https://lwn.net/Articles/1095703/>)

Author: jake

Published: 2026-09-21T14:43:18Z

Content type: release

Language: en

Sources: [LWN.net](<https://devfeed.tech/sources/lwn-net.md>)

Topics: [kernels](<https://devfeed.tech/topics/kernels.md>), [Kernel](<https://devfeed.tech/topics/kernel.md>)

Tags: [kernel](<https://devfeed.tech/tags/kernel.md>), [kernels](<https://devfeed.tech/tags/kernels.md>), [stable](<https://devfeed.tech/tags/stable.md>), [update](<https://devfeed.tech/tags/update.md>)

### AI overview

Greg Kroah-Hartman released stable versions 7.2.7, 6.18.53, and 6.12.111 of the Linux kernel, containing fixes throughout the kernel tree. Users are advised to update.

### Source excerpt

Greg Kroah-Hartman has released the 7.2.7, 6.18.53, and 6.12.111 stable kernels. As is usual these days, they are quite large; also no surprise is that they provide many important fixes throughout the kernel tree. Users of those kernels are advised to update.

## More than 9,000 patches total in the seven stable kernels for Monday

DevFeed: [More than 9,000 patches total in the seven stable kernels for Monday](<https://devfeed.tech/articles/more-than-9-000-patches-total-in-the-seven-stable-kernels-for-monday-52615.md>)

Original publisher: [Read original article](<https://blog.adafruit.com/2026/09/15/more-than-9000-patches-total-in-the-seven-stable-kernels-for-monday/>)

Author: Anne Barela

Published: 2026-09-15T19:23:51Z

Content type: news

Language: en

Sources: [Adafruit Industries](<https://devfeed.tech/sources/adafruit-industries-makers-hackers-artists-designers-and-engineers.md>)

Topics: [kernels](<https://devfeed.tech/topics/kernels.md>), [Kernel](<https://devfeed.tech/topics/kernel.md>), [patches](<https://devfeed.tech/topics/patches.md>), [releases](<https://devfeed.tech/topics/releases.md>), [upgrade](<https://devfeed.tech/topics/upgrade.md>)

Tags: [kernel](<https://devfeed.tech/tags/kernel.md>), [kernels](<https://devfeed.tech/tags/kernels.md>), [linux](<https://devfeed.tech/tags/linux.md>), [operating-system](<https://devfeed.tech/tags/operating-system.md>), [patches](<https://devfeed.tech/tags/patches.md>), [release](<https://devfeed.tech/tags/release.md>), [releases](<https://devfeed.tech/tags/releases.md>), [software](<https://devfeed.tech/tags/software.md>), [upgrade](<https://devfeed.tech/tags/upgrade.md>)

### AI overview

Seven stable Linux kernel releases were announced together, containing more than 9,000 patches in total. The 7.2.6 release includes more than 1,800 patches, and users of the listed kernel versions are advised to upgrade.

### Source excerpt

jzb at LWN reports that Greg Kroah-Hartman has announced seven stable kernel releases at once: 7.2.6, 6.18.52, 6.12.110, 6.6.157, 6.1.188, 5.15.221 and 5.10.270. Kroah-Hartman says the batch may set a record for patch count, with more than 9,000 patches across the seven releases. The 7.2.6 release alone carries more than 1,800 of them. Users of [...]

## More than 9,000 patches total in the seven stable kernels for Monday

DevFeed: [More than 9,000 patches total in the seven stable kernels for Monday](<https://devfeed.tech/articles/more-than-9-000-patches-total-in-the-seven-stable-kernels-for-monday-17389.md>)

Original publisher: [Read original article](<https://lwn.net/Articles/1093985/>)

Author: jzb

Published: 2026-09-14T13:00:07Z

Content type: news

Language: en

Sources: [LWN.net](<https://devfeed.tech/sources/lwn-net.md>)

Topics: [kernels](<https://devfeed.tech/topics/kernels.md>)

Tags: [kernels](<https://devfeed.tech/tags/kernels.md>), [upgrade](<https://devfeed.tech/tags/upgrade.md>)

### AI overview

Seven stable Linux kernels have received updates totaling more than 9,000 patches, including over 1,800 patches in version 7.2.6. Users are advised to upgrade.

### Source excerpt

Greg Kroah-Hartman has announced the 7.2.6, 6.18.52, 6.12.110, 6.6.157, 6.1.188, 5.15.221, 5.10.270 stable kernels. According to Kroah-Hartman, this batch may set a record for the number of patches with more than 9,000 in total between them. There are more than 1,800 patches in 7.2.6 alone. Users of these kernels are, of course, advised to upgrade.

## BFS File-System Being Removed For Linux 7.4

DevFeed: [BFS File-System Being Removed For Linux 7.4](<https://devfeed.tech/articles/bfs-file-system-being-removed-for-linux-7-4-12397.md>)

Original publisher: [Read original article](<https://www.phoronix.com/news/BFS-Filesystem-To-Be-Removed>)

Author: Michael Larabel

Published: 2026-09-13T11:03:26Z

Content type: news

Language: en

Sources: [Phoronix](<https://devfeed.tech/sources/phoronix.md>)

Topics: [Filesystems](<https://devfeed.tech/topics/filesystems.md>), [Linux](<https://devfeed.tech/topics/linux.md>), [kernels](<https://devfeed.tech/topics/kernels.md>), [Unix](<https://devfeed.tech/topics/unix.md>), [boot](<https://devfeed.tech/topics/boot.md>)

Tags: [boot](<https://devfeed.tech/tags/boot.md>), [bootloader](<https://devfeed.tech/tags/bootloader.md>), [desktop-linux](<https://devfeed.tech/tags/desktop-linux.md>), [filesystem](<https://devfeed.tech/tags/filesystem.md>), [filesystems](<https://devfeed.tech/tags/filesystems.md>), [kernels](<https://devfeed.tech/tags/kernels.md>), [linux](<https://devfeed.tech/tags/linux.md>), [linux-benchmarking](<https://devfeed.tech/tags/linux-benchmarking.md>), [linux-hardware-benchmarks](<https://devfeed.tech/tags/linux-hardware-benchmarks.md>), [linux-hardware-reviews](<https://devfeed.tech/tags/linux-hardware-reviews.md>), [linux-how-to](<https://devfeed.tech/tags/linux-how-to.md>), [linux-performance](<https://devfeed.tech/tags/linux-performance.md>), [linux-server-benchmarks](<https://devfeed.tech/tags/linux-server-benchmarks.md>), [open-source-graphics](<https://devfeed.tech/tags/open-source-graphics.md>), [os](<https://devfeed.tech/tags/os.md>), [phoronix](<https://devfeed.tech/tags/phoronix.md>), [phoronix-test-suite](<https://devfeed.tech/tags/phoronix-test-suite.md>), [systems](<https://devfeed.tech/tags/systems.md>), [technical](<https://devfeed.tech/tags/technical.md>), [ubuntu-benchmarks](<https://devfeed.tech/tags/ubuntu-benchmarks.md>), [ubuntu-hardware](<https://devfeed.tech/tags/ubuntu-hardware.md>)

### AI overview

BFS, the obsolete UnixWare boot-partition filesystem, is slated for removal from the Linux kernel in version 7.4. The article cites its extremely limited functionality, lack of likely users, and broader efforts to reduce maintenance burdens from rarely used filesystems.

### Source excerpt

The latest upstream file-system to be removed from the mainline Linux kernel is... BFS...

[Next page](<https://devfeed.tech/topics/kernels.md?cursor=WyIyMDI2LTA5LTEzVDExOjAzOjI2KzAwOjAwIiwgIjBjM2U1MjQxLWVlNDItNDY4ZC1iMTI4LWQ5Zjk3NDFlZGUzOCJd>)