# Transformers

Transformers is a model-definition framework for machine-learning models in text, computer vision, audio, video, and multimodal applications, supporting inference and training.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## RL Post-Training in Diffusion Transformers Is Activation Steering

DevFeed: [RL Post-Training in Diffusion Transformers Is Activation Steering](<https://devfeed.tech/articles/rl-post-training-in-diffusion-transformers-is-activation-steering-145203.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.26627>)

Author: Tuna Han Salih Meral, Kaan Oktay, Alec Helbling, Atakan Kara, Duen Horng Chau, Pinar Yanardag

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [diffusion-transformers](<https://devfeed.tech/topics/diffusion-transformers.md>), [post-training](<https://devfeed.tech/topics/post-training.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Transformers](<https://devfeed.tech/topics/transformers.md>)

Tags: [activation-steering](<https://devfeed.tech/tags/activation-steering.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [diffusion-transformers](<https://devfeed.tech/tags/diffusion-transformers.md>), [post-training](<https://devfeed.tech/tags/post-training.md>), [rl](<https://devfeed.tech/tags/rl.md>), [steering](<https://devfeed.tech/tags/steering.md>)

### AI overview

The study finds that fixed activation shifts can reproduce most reported gains from RL post-training diffusion transformers, without adding prompt-dependent transformations. The authors add steering vectors to outputs of a frozen base model and fit them to match RL-tuned checkpoints' velocity predictions, without using reward in that fitting objective. On SD3.5-M, the vectors use 0.29M parameters versus 18.8M for the RL-tuned LoRA. Despite being identical across prompts, tokens, and sampling seeds, they recover 90.0-96.3% of reward gains for six image models and 89.8% and 111.9% for two video models. In separate online RL experiments, zero-initialized vectors exceed weight-based RL's reported gains on OCR text rendering and PickScore human preference for SD3.5-M. The abstract's conclusion is that a constant shift can steer the pretrained generator toward capabilities it already computes.

### Source excerpt

arXiv:2611.26627v1 Announce Type: cross Abstract: Reinforcement learning (RL) post-training substantially improves the visual quality, text rendering, and compositional accuracy of diffusion transformers. Because capabilities such as rendering requested text or composing multiple objects are inherently prompt-dependent, a natural hypothesis is that improving them requires learning complex, input-dependent transformations. To isolate how much of these gains an input-independent shift explains, we probe each RL-tuned model with a deliberately simple parameterization: a fixed set of steering vectors added to the block outputs of the frozen base model. On SD3.5-M, these vectors hold 65x fewer parameters than the RL-tuned LoRA (0.29M against 18.8M). We fit them to match the velocity predictions of public RL-tuned checkpoints, with no reward in the objective. Although the vectors are identical for every prompt, token, and sampling seed, they recover 90.0-96.3% of the reward gains of six RL-tuned image models, and 89.8% and 111.9% on two RL-tuned video models. Initialized at zero and trained with online RL, the vectors alone exceed the reward gains of weight-based RL on text rendering (OCR) and human preference (PickScore) on SD3.5-M. A constant shift thus suffices for most of the gain of policy-gradient RL post-training: without new input-dependent transformations, it steers the pretrained generator toward what it can already compute.

## SimVLA: Attributing Gains in VLA Models Through Controlled Ablation

DevFeed: [SimVLA: Attributing Gains in VLA Models Through Controlled Ablation](<https://devfeed.tech/articles/simvla-attributing-gains-in-vla-models-through-controlled-ablation-144878.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2602.18224>)

Author: Yuankai Luo, Woping Chen, Tong Liang, Zhenguo Li

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [LLM evaluation / benchmarking](<https://devfeed.tech/topics/llm-evaluation-benchmarking.md>), [Transformers](<https://devfeed.tech/topics/transformers.md>), [enhancing AI model accuracy](<https://devfeed.tech/topics/enhancing-ai-model-accuracy.md>)

Tags: [ablation](<https://devfeed.tech/tags/ablation.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [attribution](<https://devfeed.tech/tags/attribution.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-ro](<https://devfeed.tech/tags/cs-ro.md>), [delta](<https://devfeed.tech/tags/delta.md>), [driver](<https://devfeed.tech/tags/driver.md>), [encoder](<https://devfeed.tech/tags/encoder.md>), [lightweight](<https://devfeed.tech/tags/lightweight.md>), [minimal](<https://devfeed.tech/tags/minimal.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [pretraining](<https://devfeed.tech/tags/pretraining.md>), [results](<https://devfeed.tech/tags/results.md>), [robot](<https://devfeed.tech/tags/robot.md>), [small](<https://devfeed.tech/tags/small.md>), [training](<https://devfeed.tech/tags/training.md>), [vision-language-action](<https://devfeed.tech/tags/vision-language-action.md>)

### AI overview

SimVLA's controlled ablations identify training dynamics as the largest measured driver of VLA performance: one-knob-at-a-time tests across LIBERO, CALVIN, WidowX, and Google Robot found swings of about 54-94%, while task configuration and architecture had smaller effects under matched settings. SimVLA pairs a standard vision-language backbone with a lightweight continuous-action head. Despite a small backbone and no additional large-scale robot-data VLA pretraining before benchmark-specific training, it achieved 98.6% average success on LIBERO, remained competitive on the other benchmarks, and transferred to held-out real-robot scenes. The work presents a controlled attribution study and minimal-baseline calibration, rather than a new technique.

### Source excerpt

arXiv:2602.18224v2 Announce Type: replace-cross Abstract: Vision-Language-Action (VLA) models often bundle architectural changes with different pretraining data, backbone scales, and optimization recipes, obscuring what actually drives progress. We introduce SimVLA, a deliberately minimal VLA, a standard vision-language backbone with a lightweight continuous-action head, as a controlled testbed for this attribution problem; our contribution is methodological and empirical (a controlled attribution study and a minimal-baseline calibration), not a new technique. Across LIBERO, CALVIN, WidowX, and Google Robot, one-knob-at-a-time ablations show that training dynamics are the largest measured driver, producing performance swings of $\Delta$$\approx$54--94\%, while task configuration and architecture have smaller effects under matched settings. Despite using a small backbone and no additional large-scale robot-data VLA pretraining before benchmark-specific training, SimVLA reaches 98.6\% average success on LIBERO, remains competitive across the other benchmarks, and transfers to held-out real-robot scenes. These results suggest that simple, carefully controlled VLA baselines remain underexplored and provide a calibration point for future architectural claims within the VLM-encoder plus lightweight-head family. Website: https://frontierrobo.github.io/SimVLA

## Can an LLM Fix a Language Model Without Training It?

DevFeed: [Can an LLM Fix a Language Model Without Training It?](<https://devfeed.tech/articles/can-an-llm-fix-a-language-model-without-training-it-145551.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.25613>)

Author: Bharath Kumar Bolla, Raghavender Surya Upadhayula, Vishnu Surya Reddy Nandi, Ramesh Nandipalli

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Transformers](<https://devfeed.tech/topics/transformers.md>), [LLM evaluation / benchmarking](<https://devfeed.tech/topics/llm-evaluation-benchmarking.md>), [lora](<https://devfeed.tech/topics/lora.md>)

Tags: [activation](<https://devfeed.tech/tags/activation.md>), [attention](<https://devfeed.tech/tags/attention.md>), [black-box](<https://devfeed.tech/tags/black-box.md>), [controller](<https://devfeed.tech/tags/controller.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [interactive](<https://devfeed.tech/tags/interactive.md>), [language-model](<https://devfeed.tech/tags/language-model.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [llm](<https://devfeed.tech/tags/llm.md>), [low-rank](<https://devfeed.tech/tags/low-rank.md>), [models](<https://devfeed.tech/tags/models.md>), [predictions](<https://devfeed.tech/tags/predictions.md>), [projection](<https://devfeed.tech/tags/projection.md>), [recovery](<https://devfeed.tech/tags/recovery.md>), [repair](<https://devfeed.tech/tags/repair.md>), [scripted](<https://devfeed.tech/tags/scripted.md>), [search](<https://devfeed.tech/tags/search.md>), [selection](<https://devfeed.tech/tags/selection.md>), [sessions](<https://devfeed.tech/tags/sessions.md>), [study](<https://devfeed.tech/tags/study.md>), [training](<https://devfeed.tech/tags/training.md>)

### AI overview

The study finds that an external language model can help repair selected Transformer faults by choosing numerical projection edits from predictions, activation summaries, and weight statistics, without gradient updates. Across 20 sessions and five models, live selection matched black-box search on SmolLM2 and Qwen2.5 for a two-layer fault; on Pythia it reached 65.62% completion accuracy, versus 0% for black-box search and 66.15% for a fixed projection. For a compound attention and feed-forward fault, scripted and LLM-guided edits using the same operator family raised the five-checkpoint mean from 24.89% to 71.25%. The comparisons also show that feedback does not always account for recovery: operator coverage can explain gains, supervised adaptation sometimes performs better, and scripted projections on unmodified checkpoints produced small gains or losses. These conclusions concern the tested faults and completion tasks.

### Source excerpt

arXiv:2611.25613v1 Announce Type: new Abstract: Although an external language model can select and test repairs to another model's weights, behavior can also be recovered through fixed numerical edits. We assessed the value of interactive selection in a controlled study of Transformer repair. After inspecting predictions, activation summaries, and weight statistics, the editor selects projection recipes to apply to the target without gradient updates. We analyzed 20 recorded sessions involving five masked and causal language models, comparing the selected edits with fixed projections, scripted and matched-budget black-box search, and low-rank adaptation. Under the core two-layer fault, live selection matches black-box search on SmolLM2 and Qwen2.5. For Pythia, live selection achieves 65.62% completion accuracy, compared with 0% for black-box search and 66.15% for a fixed projection. Under a separate compound fault affecting attention and feed-forward weights, both scripted and LLM-guided selection increase the descriptive five-checkpoint mean from 24.89% to 71.25% using the same operator family. On unmodified checkpoints, scripted projection yields small gains or losses on a separate set of framed questions. The experiments identify cases in which feedback changes an edit decision, operator coverage accounts for recovery, or supervised adaptation performs better. Combined with measurements of parameter restoration and preservation, these comparisons establish the scope of controller-guided repair for the tested faults and completion tasks.

## When Does Depth Matter For In-Context Learning? Adaptive Inference in Deep Transformers

DevFeed: [When Does Depth Matter For In-Context Learning? Adaptive Inference in Deep Transformers](<https://devfeed.tech/articles/when-does-depth-matter-for-in-context-learning-adaptive-inference-in-deep-transformers-144908.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2606.16694>)

Author: Ravin Raj, Gautam Reddy

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Transformers](<https://devfeed.tech/topics/transformers.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Inference](<https://devfeed.tech/topics/inference.md>)

Tags: [adaptive](<https://devfeed.tech/tags/adaptive.md>), [context](<https://devfeed.tech/tags/context.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [distributed-inference](<https://devfeed.tech/tags/distributed-inference.md>), [gaussian](<https://devfeed.tech/tags/gaussian.md>), [in-context-learning](<https://devfeed.tech/tags/in-context-learning.md>), [inference](<https://devfeed.tech/tags/inference.md>), [learning](<https://devfeed.tech/tags/learning.md>), [linear-attention](<https://devfeed.tech/tags/linear-attention.md>), [linear-regression](<https://devfeed.tech/tags/linear-regression.md>), [patching](<https://devfeed.tech/tags/patching.md>), [physics-app-ph](<https://devfeed.tech/tags/physics-app-ph.md>), [q-bio-nc](<https://devfeed.tech/tags/q-bio-nc.md>), [routing](<https://devfeed.tech/tags/routing.md>), [structure](<https://devfeed.tech/tags/structure.md>), [transformers](<https://devfeed.tech/tags/transformers.md>)

### AI overview

For in-context linear regression, the theory predicts that a single transformer layer without feedforward blocks is optimal for Gaussian priors, while deeper transformers outperform it for non-Gaussian, tree-like priors. The authors propose that successive layers use intermediate "function vectors" to infer a latent context variable at finer scales, enabling adaptive information routing. Trained linear attention transformers reproduce the theory's quantitative predictions, and causal key-patching experiments support a role for intermediate function vectors in that routing. The results suggest depth and feedforward blocks are advantageous when latent variables have hierarchical structure.

### Source excerpt

arXiv:2606.16694v3 Announce Type: replace Abstract: Transformers perform computations through many successive attention and feedforward blocks, allowing them to learn complex correlations between a large collection of coupled variables. When does stacking successive attention-feedforward computations over many layers provide a computational advantage over a single transformer block? We address this question by examining in-context learning in generalized linear attention transformers. We first introduce a general theory of distributed inference in such transformers, subject to constraints on communication and depth. We show that such systems can exploit internal representations (`function vectors') to infer a latent context variable at increasingly finer scales over its layers. For an in-context linear regression task, the theory predicts that while one-layer transformers without feedforward blocks are optimal for Gaussian priors over the context variable, multi-layer transformers are superior for non-Gaussian, tree-like priors. Trained linear attention transformers reproduce quantitative predictions from the theory. Using causal key-patching experiments, we verify that function vectors in intermediate layers mediate adaptive routing of information. Our results suggest that depth and feedforward blocks enable transformers to implement adaptive inference, and this is advantageous when the distribution over latent variables has hierarchical structure.

## Is a Description Worth a Thousand Demonstrations? Specificity and Reliability of Task Descriptions in In-Context Learning

DevFeed: [Is a Description Worth a Thousand Demonstrations? Specificity and Reliability of Task Descriptions in In-Context Learning](<https://devfeed.tech/articles/is-a-description-worth-a-thousand-demonstrations-specificity-and-reliability-of-task-descriptions-in-in-context-learning-145070.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.25560>)

Author: Bruce Nguyen, Bei Jiang, Linglong Kong

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [few-shot](<https://devfeed.tech/topics/few-shot.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Transformers](<https://devfeed.tech/topics/transformers.md>), [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>)

Tags: [bayes](<https://devfeed.tech/tags/bayes.md>), [bayesian](<https://devfeed.tech/tags/bayesian.md>), [bayesian-statistics](<https://devfeed.tech/tags/bayesian-statistics.md>), [computer-science](<https://devfeed.tech/tags/computer-science.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [in-context-learning](<https://devfeed.tech/tags/in-context-learning.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [linear-regression](<https://devfeed.tech/tags/linear-regression.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [prompt](<https://devfeed.tech/tags/prompt.md>), [reasoning](<https://devfeed.tech/tags/reasoning.md>), [reliability](<https://devfeed.tech/tags/reliability.md>), [sample-size](<https://devfeed.tech/tags/sample-size.md>), [specificity](<https://devfeed.tech/tags/specificity.md>), [training](<https://devfeed.tech/tags/training.md>), [transformers](<https://devfeed.tech/tags/transformers.md>)

### AI overview

The paper models task descriptions in in-context learning as informative priors and uses effective sample size (ESS) to express their predictive value in equivalent demonstrations. In its linear regression analysis, a reliable description's ESS increases with specificity and task dimension; if a description may be irrelevant, specificity adds little without demonstrations. Experiments find that meta-trained Transformers approach Bayes-optimal ESS on their training distribution. Pretrained language models tend to underuse highly specific descriptions without reasoning, while most closely match the Bayes-optimal predictor when reasoning is enabled.

### Source excerpt

arXiv:2611.25560v1 Announce Type: new Abstract: A prompt for in-context learning (ICL) can describe a task in addition to, or instead of, demonstrating it through examples. Yet how task descriptions work remains poorly understood theoretically. Through the Bayesian view of ICL, we model descriptions as informative priors characterized by their specificity and reliability. We adapt the effective sample size (ESS) from Bayesian statistics, quantifying a description's predictive value as an equivalent number of demonstrations. For in-context linear regression, we show that a reliable description's ESS grows with its specificity and with the task's dimension. A description that may be irrelevant, however, gains little from its specificity unless demonstrations are given. In practice, do in-context learners use descriptions as a Bayes-optimal predictor would? Our experiments show that meta-trained Transformers come close to the Bayes-optimal ESS when tested on their training distribution. Pretrained language models, however, make too little use of highly specific descriptions when answering without reasoning; most of them closely match the Bayes-optimal predictor only when reasoning is enabled. These results extend the Bayesian analysis of ICL from demonstrations to descriptions.

## Friction-Augmented Drifting Models for Resource-Efficient Domain Translation

DevFeed: [Friction-Augmented Drifting Models for Resource-Efficient Domain Translation](<https://devfeed.tech/articles/friction-augmented-drifting-models-for-resource-efficient-domain-translation-144887.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2604.18194>)

Author: Arkadii Kazanskii, Tatiana Petrova, Andrey Ustyuzhanin, Konstantin Bagrianskii, Aleksandr Puzikov, Radu State

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Transformers](<https://devfeed.tech/topics/transformers.md>)

Tags: [cost](<https://devfeed.tech/tags/cost.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [domain](<https://devfeed.tech/tags/domain.md>), [drift](<https://devfeed.tech/tags/drift.md>), [flow-matching](<https://devfeed.tech/tags/flow-matching.md>), [generator](<https://devfeed.tech/tags/generator.md>), [gradient-descent](<https://devfeed.tech/tags/gradient-descent.md>), [learning](<https://devfeed.tech/tags/learning.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [models](<https://devfeed.tech/tags/models.md>), [translation](<https://devfeed.tech/tags/translation.md>)

### AI overview

DMF scales a Drifting Model's kernel-drift displacement linearly from full drift to zero. Under plain gradient descent, this matches linear learning-rate decay; with Adam, it also changes gradient moments, and a constant multiplier c has the exact-arithmetic effect of replacing ε with ε/c. In a two-particle model with unnormalised Laplace interactions, the authors derive where attraction balances repulsion. For FFHQ adult-to-child translation, DMF had lower FID and CMMD than fixed-rate DM in all five seed pairs. A separate comparison favored linear learning-rate decay on mean FID and CMMD, though both 95% intervals for paired differences included zero. At batch size 32 on one RTX A6000, mean training times were 6.85 minutes for DMF, 6.93 for DM, and 133.48 for OFM; DMF also had lower FID and CMMD than OFM, with paired p-values 0.002 and 0.021.

### Source excerpt

arXiv:2604.18194v3 Announce Type: replace Abstract: Drifting Models (DMs) train a one-step generator by regressing its output towards a target displaced by a kernel drift. DMF (Drifting Model with Friction) scales this displacement linearly from full drift to zero. With plain gradient descent, scaling the drift gives the same updates as linear learning-rate decay. With Adam, it also changes the gradient moments. For a constant multiplier $c>0$, its effect in exact arithmetic is to replace Adam's $\epsilon$ by $\epsilon/c$. In a two-particle model with unnormalised Laplace interactions, we derive the distance at which attraction balances repulsion. On FFHQ adult-to-child translation, DMF lowered FID and CMMD relative to fixed-rate DM in all five seed pairs. In a separate paired comparison, mean FID and CMMD were lower for DM with linear learning-rate decay than for DMF. Both 95% intervals for the paired differences included zero. DMF inherits DM's low training cost. At batch size 32 on one RTX A6000, mean training times were 6.85 minutes for DMF, 6.93 for DM and 133.48 for Optimal Flow Matching (OFM). At this batch size, DMF also achieved lower FID and CMMD than OFM (paired $p=0.002$ and $0.021$, respectively).

## NVIDIA's Dory Shows Why Thinking Longer Doesn't Make AI Smarter

DevFeed: [NVIDIA's Dory Shows Why Thinking Longer Doesn't Make AI Smarter](<https://devfeed.tech/articles/nvidia-s-dory-shows-why-thinking-longer-doesn-t-make-ai-smarter-142901.md>)

Original publisher: [Read original article](<https://hackernoon.com/nvidias-dory-shows-why-thinking-longer-doesnt-make-ai-smarter?source=rss>)

Author: Michał Piszczek

Published: 2026-11-19T08:30:28Z

Content type: opinion

Language: en

Sources: [HackerNoon](<https://devfeed.tech/sources/hackernoon.md>)

Topics: [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [Transformers](<https://devfeed.tech/topics/transformers.md>), [Language models](<https://devfeed.tech/topics/language-models.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-agents](<https://devfeed.tech/tags/ai-agents.md>), [ai-thinking](<https://devfeed.tech/tags/ai-thinking.md>), [arithmetic](<https://devfeed.tech/tags/arithmetic.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [chain-of-thought](<https://devfeed.tech/tags/chain-of-thought.md>), [code](<https://devfeed.tech/tags/code.md>), [compute](<https://devfeed.tech/tags/compute.md>), [confidence-interval](<https://devfeed.tech/tags/confidence-interval.md>), [cost](<https://devfeed.tech/tags/cost.md>), [costs](<https://devfeed.tech/tags/costs.md>), [decision-model](<https://devfeed.tech/tags/decision-model.md>), [effort](<https://devfeed.tech/tags/effort.md>), [experiment](<https://devfeed.tech/tags/experiment.md>), [failure](<https://devfeed.tech/tags/failure.md>), [fetch](<https://devfeed.tech/tags/fetch.md>), [future-of-ai](<https://devfeed.tech/tags/future-of-ai.md>), [generative-ai](<https://devfeed.tech/tags/generative-ai.md>), [llms](<https://devfeed.tech/tags/llms.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [nvidia](<https://devfeed.tech/tags/nvidia.md>), [paying-ai-to-think-harder](<https://devfeed.tech/tags/paying-ai-to-think-harder.md>), [thinking](<https://devfeed.tech/tags/thinking.md>), [transformers](<https://devfeed.tech/tags/transformers.md>)

### AI overview

NVIDIA's 4B Dory transformer makes extra reasoning a measurable choice: it repeats a 20-layer core while keeping the same weights, increasing computation per token without adding information. The excerpt says more passes can help assemble facts already in the input, but cannot supply missing facts; use an external lookup for those and code or a calculator for exact arithmetic or strings. A cited Fastino 27B result reports a small held-out gain (+0.56 points; 95% CI 0.27-0.89) alongside about 35% higher median latency (83 to 112 ms), and a third pass worsened one causal-reasoning benchmark. The suggested checks are whether the answer is present, where gains flatten, and the cost per verified answer including retries. The excerpt distinguishes transformer loops, generated chain of thought, tool-using agent loops, and across-version recursive self-improvement by what they consume or leave behind; only tool use brings in outside information.

### Source excerpt

Looped transformers turn AI thinking into a dial. The Alice test shows when turning it up helps and when you pay for effort that can't succeed.

## LoopCache: Accelerating Long-Context Self-Speculative Decoding for Looped Transformers

DevFeed: [LoopCache: Accelerating Long-Context Self-Speculative Decoding for Looped Transformers](<https://devfeed.tech/articles/loopcache-accelerating-long-context-self-speculative-decoding-for-looped-transformers-143503.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.24301>)

Author: Zizhuo Fu, Haochen Huang, Shengxuan Qiu, Linye Wei, Ziyu Tang, Runsheng Wang, Meng Li

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Model Optimizer](<https://devfeed.tech/topics/model-optimizer.md>), [Transformers](<https://devfeed.tech/topics/transformers.md>), [Decoding](<https://devfeed.tech/topics/decoding.md>)

Tags: [cache](<https://devfeed.tech/tags/cache.md>), [confidence](<https://devfeed.tech/tags/confidence.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [decoding](<https://devfeed.tech/tags/decoding.md>), [kv-cache](<https://devfeed.tech/tags/kv-cache.md>), [long-context](<https://devfeed.tech/tags/long-context.md>), [speculative-decoding](<https://devfeed.tech/tags/speculative-decoding.md>), [tokens](<https://devfeed.tech/tags/tokens.md>), [transformers](<https://devfeed.tech/tags/transformers.md>)

### AI overview

LoopCache targets two bottlenecks in wave-style self-speculative decoding for long-context Looped Transformers: fixed-loop drafts can become stale as predictions change, and batching tokens across loops can load up to R times the KV cache per step. It dynamically updates drafts using output confidence and rotates loop-specific KV caches so each step accesses one loop's cache while batching at the nearest matching cycle for lossless verification. It also retains independent caches for selected loop-sensitive shallow layers to improve acceptance rates with negligible overhead. The abstract reports average speedups of 10.25x over WFD and 14.56x over LoopSpec.

### Source excerpt

arXiv:2611.24301v1 Announce Type: cross Abstract: Looped Transformers achieve large effective depth with compact models by repeatedly applying a weight-shared block across R recurrent steps, each maintaining a loop-specific key-value (KV) cache. To accelerate sequential loop execution, recent wave-style self-speculative methods like WaveFront Decoding (WFD) and LoopSpec pipeline draft and target tokens across loops in a single step. However, they face two critical limitations. First, they draft only at fixed loops and ignore intermediate prediction changes, wasting computation on stale trajectories. Second, batching tokens from different loops loads up to Rx the KV cache per step, severely amplifying memory traffic in long-context scenarios. To address these challenges, we propose LoopCache, an efficient self-speculative decoding framework for long-context Looped Transformers. Based on the correlation between prediction switches and output confidence, LoopCache introduces confidence-guided dynamic draft updating to refine drafts in flight across all intermediate loops. Furthermore, LoopCache employs a cyclic loop KV cache strategy that rotates through loop-specific KV caches, restricting per-step KV traffic to a single loop while batching tokens at the nearest matching cycle for lossless verification. By also selectively retaining independent KV caches for a few loop-sensitive shallow layers, LoopCache improves acceptance rates with negligible overhead. Experiments show that LoopCache achieves average 10.25x and 14.56x speedup over WFD and LoopSpec.

## PEARL: Personalized Streaming Video Understanding with Runtime-Defined Concepts

DevFeed: [PEARL: Personalized Streaming Video Understanding with Runtime-Defined Concepts](<https://devfeed.tech/articles/pearl-personalized-streaming-video-understanding-with-runtime-defined-concepts-142515.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2603.20422>)

Author: Yuanhong Zheng, Ruichuan An, Xiaopeng Lin, Yuxing Liu, Sihan Yang, Huanyu Zhang, Haodong Li, Qintong Zhang, Renrui Zhang, Guopeng Li, Yifan Zhang, Yuheng Li, Wentao Zhang

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Transformers](<https://devfeed.tech/topics/transformers.md>), [Streaming](<https://devfeed.tech/topics/streaming.md>), [object-detection](<https://devfeed.tech/topics/object-detection.md>)

Tags: [actions](<https://devfeed.tech/tags/actions.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-ir](<https://devfeed.tech/tags/cs-ir.md>), [history](<https://devfeed.tech/tags/history.md>), [streaming](<https://devfeed.tech/tags/streaming.md>), [streaming-video](<https://devfeed.tech/tags/streaming-video.md>), [user](<https://devfeed.tech/tags/user.md>), [video](<https://devfeed.tech/tags/video.md>), [vision-language-models-vlms](<https://devfeed.tech/tags/vision-language-models-vlms.md>)

### AI overview

PEARL targets personalized streaming video understanding: it lets users define concepts during a video stream and answer questions about their current state or earlier appearances. It stores concept definitions separately from archived clips, then rewrites queries with concept descriptions to retrieve relevant history. The authors introduce PSVU-Bench, with 132 videos and timestamped questions covering concept definition, real-time QA, and past-time QA, and PSVU-10K to train PEARL's retriever and answering VLM once; concepts defined at test time need no training. Across three architectures, PEARL without additional training improves real-time and past-time QA by averages of 15.05% and 12.60% over corresponding offline models, and surpasses existing online video models on both. Training further improves results on Qwen3-VL-8B by 8.66% and 9.39%, respectively.

### Source excerpt

arXiv:2603.20422v2 Announce Type: replace-cross Abstract: Unlike humans, who learn new concepts from a continuous visual stream, existing personalized vision-language models (VLMs) internalize a predefined concept set during training and operate on static images or offline videos. They cannot register a concept that a user defines in the middle of a stream, nor recall the earlier moments in which it appeared. We formalize this task as Personalized Streaming Video Understanding (PSVU) and introduce PSVU-Bench, the first benchmark for this task, which annotates 132 videos with timestamped questions. Its Concept-Definition, Real-Time and Past-Time QA ask a model to register concepts at arbitrary timestamps, recognize their current state, including user-defined actions, and retrieve their historical evidence. We propose PEARL, a strong plug-and-play baseline that stores user-defined concepts apart from archived video clips and rewrites each query with concept descriptions to retrieve the relevant history. Grounding a named concept in video is a skill shared by all concepts. We therefore construct PSVU-10K, a training set of verified concept questions on public internet videos, and train the retriever and answering VLM of PEARL on it once. Concepts defined at test time still require no training. Applied to three architectures without training, PEARL improves Real-Time QA by 15.05% and Past-Time QA by 12.60% on average over the corresponding offline models, and all three surpass existing online video models on both QA types. Training further improves PEARL on Qwen3-VL-8B by 8.66% on Real-Time QA and 9.39% on Past-Time QA, with gains in every category. Code, data and models are available at https://github.com/Yuanhong-Zheng/PEARL.

## Variable Length Diffuse Everything: Arbitrary Length Generative Modeling on Arbitrary State Spaces

DevFeed: [Variable Length Diffuse Everything: Arbitrary Length Generative Modeling on Arbitrary State Spaces](<https://devfeed.tech/articles/variable-length-diffuse-everything-arbitrary-length-generative-modeling-on-arbitrary-state-spaces-143433.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.23684>)

Author: Kevin Rojas, Felix X. -F. Ye, Molei Tao

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Transformers](<https://devfeed.tech/topics/transformers.md>)

Tags: [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [data](<https://devfeed.tech/tags/data.md>), [diffusion](<https://devfeed.tech/tags/diffusion.md>), [effectiveness](<https://devfeed.tech/tags/effectiveness.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [flow](<https://devfeed.tech/tags/flow.md>), [framework](<https://devfeed.tech/tags/framework.md>), [generative](<https://devfeed.tech/tags/generative.md>), [generator](<https://devfeed.tech/tags/generator.md>), [layout](<https://devfeed.tech/tags/layout.md>), [modeling](<https://devfeed.tech/tags/modeling.md>), [models](<https://devfeed.tech/tags/models.md>), [multimodal](<https://devfeed.tech/tags/multimodal.md>), [padding](<https://devfeed.tech/tags/padding.md>), [stat-ml](<https://devfeed.tech/tags/stat-ml.md>), [training](<https://devfeed.tech/tags/training.md>), [variable](<https://devfeed.tech/tags/variable.md>)

### AI overview

Variable Length Diffuse Everything (VL-DE) models variable-length multimodal data on arbitrary state spaces using an insertion-based mechanism instead of padding or a specialized length objective. The authors say this yields a tractable training objective. They also develop a higher-order sampler that accounts for time dependence in VL-DE's additive-form generator. Experiments on layout and molecule generation are reported, though the supplied excerpt gives no quantitative results.

### Source excerpt

arXiv:2611.23684v1 Announce Type: new Abstract: Diffusion and flow-based generative models have achieved remarkable results across diverse applications, but most assume data of fixed-dimension/length. Meanwhile, real-world problems such as layout generation often involve variable-length, multimodal data, such as continuous bounding box coordinates paired with discrete types. Existing methods typically address variable length by padding data to a fixed size or by introducing specialized objectives to model lengths explicitly, adding complexity and limiting generality. We propose Variable Length Diffuse Everything (VL-DE), a simple and general framework for variable-length multimodal generation on arbitrary state spaces. Instead of padding, VL-DE handles variable length through an insertion-based mechanism grounded in a principled theoretical framework, yielding a tractable training objective. In addition, by utilizing the fact that VL-DE is based on a generator with an additive form, we develop a higher-order sampler by appropriately handling the time dependence. Extensive experiments on layout and molecule generation demonstrate the effectiveness of our approach.

## Stability Does Not Mirror Plasticity: A Module-Level View of Declarative and Procedural Learning in Transformers

DevFeed: [Stability Does Not Mirror Plasticity: A Module-Level View of Declarative and Procedural Learning in Transformers](<https://devfeed.tech/articles/stability-does-not-mirror-plasticity-a-module-level-view-of-declarative-and-procedural-learning-in-transformers-143391.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.23427>)

Author: Jinjun Peng, Wailiang Zhao, Baishakhi Ray, Zhou Yu, Jufeng Yang

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Transformers](<https://devfeed.tech/topics/transformers.md>), [model architecture](<https://devfeed.tech/topics/model-architecture.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [all](<https://devfeed.tech/tags/all.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [best](<https://devfeed.tech/tags/best.md>), [capability](<https://devfeed.tech/tags/capability.md>), [continual-learning](<https://devfeed.tech/tags/continual-learning.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [declarative](<https://devfeed.tech/tags/declarative.md>), [echo](<https://devfeed.tech/tags/echo.md>), [environment](<https://devfeed.tech/tags/environment.md>), [existing](<https://devfeed.tech/tags/existing.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [explain](<https://devfeed.tech/tags/explain.md>), [functional](<https://devfeed.tech/tags/functional.md>), [learning](<https://devfeed.tech/tags/learning.md>), [mirror](<https://devfeed.tech/tags/mirror.md>), [module](<https://devfeed.tech/tags/module.md>), [plasticity](<https://devfeed.tech/tags/plasticity.md>), [procedural](<https://devfeed.tech/tags/procedural.md>), [transformers](<https://devfeed.tech/tags/transformers.md>), [view](<https://devfeed.tech/tags/view.md>)

### AI overview

In controlled synthetic experiments, plasticity and stability in transformers depend on different aspects of module function. QK and VO projections were most plastic for procedural tasks, while MLPs were most plastic for declarative memorization. During learning of a new procedure, updating QK preserved over half of stored declarative memory; updating MLPs preserved much less, and updating VO lost roughly four-fifths. Reset-and-recovery experiments suggest plasticity reflects a module's fit for a task's main computation or storage, while stability also depends on how existing capabilities use that module, including to access information stored elsewhere. Experiments on a realistic model and tasks broadly echoed these patterns.

### Source excerpt

arXiv:2611.23427v1 Announce Type: new Abstract: Plasticity and stability in continual learning are typically studied at the entire-model level and in a task-agnostic manner, yet prior work has reported divergent conclusions across settings. Inspired by cognitive distinctions between declarative and procedural memory, we instead study how plasticity and stability vary across transformer modules and task types. In a controlled synthetic environment with clean measurement, we show a double dissociation: query-key (QK) and value-output (VO) projections are the most plastic on procedural tasks, whereas MLPs are the most plastic on declarative memorization. Stability, however, does not simply mirror plasticity: when learning a new procedure, updating QK retains over half of the stored declarative memory, while updating MLPs retains far less and updating VO loses roughly four-fifths. Targeted module reset-and-recovery experiments explain this asymmetry: plasticity tracks which module is best suited to a task's primary computation or storage, whereas stability depends on all ways the existing capability relies on that module, including access to information stored in other modules. Experiments on a realistic model and tasks broadly echo these asymmetric dynamics. Overall, our work shows that plasticity and stability depend on different aspects of a module's functional role, offering a task-dependent, module-level perspective on continual learning.

## When Does Continual Pre-Training Help? A Controlled Study of Dialectal Adaptation for Qu\\'ebec French

DevFeed: [When Does Continual Pre-Training Help? A Controlled Study of Dialectal Adaptation for Qu\\'ebec French](<https://devfeed.tech/articles/when-does-continual-pre-training-help-a-controlled-study-of-dialectal-adaptation-for-qu-ebec-french-142823.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.24699>)

Author: Eeham Khan, Richard Khoury, Leila Kosseim

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Transformers](<https://devfeed.tech/topics/transformers.md>), [Localization (l10n)](<https://devfeed.tech/topics/localization.md>)

Tags: [aggregate](<https://devfeed.tech/tags/aggregate.md>), [analysis](<https://devfeed.tech/tags/analysis.md>), [assessment](<https://devfeed.tech/tags/assessment.md>), [benefit](<https://devfeed.tech/tags/benefit.md>), [budget](<https://devfeed.tech/tags/budget.md>), [capability](<https://devfeed.tech/tags/capability.md>), [composition](<https://devfeed.tech/tags/composition.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [data](<https://devfeed.tech/tags/data.md>), [data-volume](<https://devfeed.tech/tags/data-volume.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [existing](<https://devfeed.tech/tags/existing.md>), [experiment](<https://devfeed.tech/tags/experiment.md>), [expression](<https://devfeed.tech/tags/expression.md>), [improvements](<https://devfeed.tech/tags/improvements.md>), [language](<https://devfeed.tech/tags/language.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [llms](<https://devfeed.tech/tags/llms.md>), [model](<https://devfeed.tech/tags/model.md>), [study](<https://devfeed.tech/tags/study.md>), [training](<https://devfeed.tech/tags/training.md>)

### AI overview

A controlled study across nine language models found that continual pre-training for Québec French helps mainly when a model has limited existing competence in the dialect. Under a matched 30-million-token budget, corpus composition mattered more than volume, with literary text accounting for most gains. Adaptation primarily improved understanding of regional expressions, not grammatical competence; corpus register shaped which expressions were learned. The study also found capability-specific trade-offs, including reading-comprehension declines that aggregate scores could hide. It recommends assessing base-model competence, analyzing training sources, and evaluating specific tasks before treating CPT as an adaptation strategy.

### Source excerpt

arXiv:2611.24699v1 Announce Type: cross Abstract: Continual pre-training (CPT) is widely used to adapt pretrained language models to dialectal and non-standard language varieties, yet it remains unclear when this additional training is necessary, what drives its benefits, and whether aggregate evaluation scores adequately capture its effects. In this paper, we present a controlled study of CPT for Qu\'ebec French language understanding across nine LLMs spanning different families and scales. Using a budget-matched protocol of 30M tokens per experiment, our results show that the benefit of CPT depends strongly on the starting model's existing competence in the target variety. When CPT is beneficial, the improvements are driven more by corpus composition than by data volume, with literary text accounting for most of the gain. Paired Qu\'ebec and standard French evaluations further show that CPT primarily improves expression comprehension rather than grammatical competence, and that corpus register determines which regional expressions are acquired. These gains also involve capability-specific trade-offs, including declines in reading comprehension that aggregate scores can conceal. Overall, these findings show that CPT for dialectal adaptation should not be treated as a uniformly beneficial default. Instead, adaptation strategies should begin with a careful assessment of the base model's existing competence, accompanied by source-level data analysis and task-level evaluation.

## torchtest2: One-Call Sanity Checks for PyTorch Models,Evaluated on Real and Unseen Architectures

DevFeed: [torchtest2: One-Call Sanity Checks for PyTorch Models,Evaluated on Real and Unseen Architectures](<https://devfeed.tech/articles/torchtest2-one-call-sanity-checks-for-pytorch-models-evaluated-on-real-and-unseen-architectures-142145.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.23273>)

Author: Grzegorz Gwardys

Published: 2026-11-19T05:00:00Z

Content type: news

Language: en

Sources: [arXiv: Software Engineering](<https://devfeed.tech/sources/arxiv-software-engineering.md>)

Topics: [PyTorch](<https://devfeed.tech/topics/pytorch.md>), [Test-driven development](<https://devfeed.tech/topics/tdd.md>), [Transformers](<https://devfeed.tech/topics/transformers.md>), [timm](<https://devfeed.tech/topics/timm.md>)

Tags: [17](<https://devfeed.tech/tags/17.md>), [3](<https://devfeed.tech/tags/3.md>), [78](<https://devfeed.tech/tags/78.md>), [abstract](<https://devfeed.tech/tags/abstract.md>), [after](<https://devfeed.tech/tags/after.md>), [all](<https://devfeed.tech/tags/all.md>), [also](<https://devfeed.tech/tags/also.md>), [analysis](<https://devfeed.tech/tags/analysis.md>), [approach](<https://devfeed.tech/tags/approach.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [architectures](<https://devfeed.tech/tags/architectures.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [attention](<https://devfeed.tech/tags/attention.md>), [batch](<https://devfeed.tech/tags/batch.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [bootstrap](<https://devfeed.tech/tags/bootstrap.md>), [call](<https://devfeed.tech/tags/call.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-se](<https://devfeed.tech/tags/cs-se.md>), [models](<https://devfeed.tech/tags/models.md>), [one](<https://devfeed.tech/tags/one.md>), [pytorch](<https://devfeed.tech/tags/pytorch.md>)

### AI overview

torchtest2 runs 18 PyTorch sanity checks on one batch with a single call, restores touched state, and uses plugins to adapt to model families. On 191 models from unseen architecture families, it detected 78% of injected bugs (95% CI 75-80%), compared with 40% for the original torchtest checks and 48% for a training monitor. It also found a batch-coupling bug in timm's GCViT, while real models revealed false alarms missed by the tool's tests and random architectures. Its reach is limited: only 3 of 28 reproduced Stack Overflow bugs and 3 of 18 TrainCheck silent errors were detectable by a pre-training tester. A median check took 1.0 second.

### Source excerpt

arXiv:2611.23273v1 Announce Type: new Abstract: Many bugs in deep learning code are silent: a layer that is never called, an optimizer created before the classification head was replaced, a recurrent layer that treats the batch as time, or an ignored attention mask. The code runs, the loss decreases somewhat, and the defect surfaces, if ever, after days of GPU time. We present torchtest2, an open-source library that runs 18 sanity checks on a single batch with one call, restores the PyTorch state it touches, and adapts to model families through plugins. We evaluate it by mutation analysis with 17 classes of silent bugs on 103 models from torchvision, timm and Hugging Face transformers, on 600 random architectures and on 20 hand-written realistic bugs, against the checks of the original torchtest, overfitting a single batch, a smoke test and a training monitor in the style of AUTOTRAINER and DeepDiagnosis. Every mutant is compared with its healthy counterpart, and intervals come from a bootstrap over models. After freezing the tool, we ran it on 191 models of architecture families it had never seen: counting, for every tool alike, a failing report where the healthy model's report does not fail, it detected 78% of the injected bugs (95% CI 75-80), against 40% for the original torchtest checks and 48% for the training monitor. On real models its batch-independence check found a bug in timm's GCViT that couples the images of a batch. Real models also exposed false alarms that neither the tool's own test suite nor random architectures had revealed. Two independent collections of real silent bugs mark the limits of the approach: only 3 of 28 reproduced PyTorch bugs from Stack Overflow and 3 of the 18 silent training errors released with TrainCheck are within reach of a pre-training model tester. A median check takes 1.0 s. We release the library, the benchmark and all raw results.

## AdaS-ViT: Test-Time Adaptive Routing for Spiking Vision Transformers with Ultra-High Energy Efficiency

DevFeed: [AdaS-ViT: Test-Time Adaptive Routing for Spiking Vision Transformers with Ultra-High Energy Efficiency](<https://devfeed.tech/articles/adas-vit-test-time-adaptive-routing-for-spiking-vision-transformers-with-ultra-high-energy-efficiency-142696.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.23728>)

Author: Zecheng Hao, Zihan Huang, Shenghao Xie, Jiyuan Zhang, Zhaofei Yu, Tiejun Huang

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Transformer](<https://devfeed.tech/topics/transformer.md>), [image recognition](<https://devfeed.tech/topics/image-recognition.md>), [Green Software](<https://devfeed.tech/topics/green-software.md>), [Transformers](<https://devfeed.tech/topics/transformers.md>), [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [adaptive](<https://devfeed.tech/tags/adaptive.md>), [all](<https://devfeed.tech/tags/all.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [attention](<https://devfeed.tech/tags/attention.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [boundary](<https://devfeed.tech/tags/boundary.md>), [building](<https://devfeed.tech/tags/building.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-ne](<https://devfeed.tech/tags/cs-ne.md>), [efficiency](<https://devfeed.tech/tags/efficiency.md>), [energy](<https://devfeed.tech/tags/energy.md>), [energy-efficiency](<https://devfeed.tech/tags/energy-efficiency.md>), [routing](<https://devfeed.tech/tags/routing.md>), [test](<https://devfeed.tech/tags/test.md>), [time](<https://devfeed.tech/tags/time.md>), [transformers](<https://devfeed.tech/tags/transformers.md>), [vision](<https://devfeed.tech/tags/vision.md>), [vision-transformers](<https://devfeed.tech/tags/vision-transformers.md>), [vit](<https://devfeed.tech/tags/vit.md>), [with](<https://devfeed.tech/tags/with.md>)

### AI overview

AdaS-ViT reduces redundant spiking computation by adapting firing precision at test time across temporal, spatial and channel dimensions. Its AdaE-IF module uses attention scores to route salient regions to fine-grained firing patterns and the rest to coarser, lower-energy patterns; S-GSA and S-LGU add selective spike emission at token and channel granularity. A progressive spatio-temporal training curriculum is used to stabilize the routing masks and firing precision. The abstract reports competitive results across benchmarks and 1.02-2.45 mJ per image on ImageNet-1k, with an accuracy-per-energy ratio of 32x-72x.

### Source excerpt

arXiv:2611.23728v1 Announce Type: cross Abstract: Spiking Neural Networks (SNNs) promise event-driven and energy-efficient vision, yet existing spiking vision transformers (S-ViTs) largely confine spiking tokens to fixed and uniform firing schemes along the temporal, spatial and channel dimensions, whereas suppressing the computational redundancy thus incurred calls for adaptive firing mechanisms that operate jointly across all three axes at test time. To this end, we propose the Adaptive Efficient Integrate-and-Fire (AdaE-IF) model, which consults the attention score of each spiking token and routes salient regions to fine-grained firing patterns while assigning energy-frugal coarse patterns to the remaining regions. Building on AdaE-IF, we further introduce Spiking Gated Self-Attention (S-GSA) and Spiking Local-Gated Unit (S-LGU), which enforce selective spike emission at token and channel granularity. We integrate the aforementioned mechanisms and modules into AdaS-ViT, a test-time adaptive-routing architecture trained with a progressive spatio-temporal curriculum that stabilizes the co-adaptation of firing precision and routing masks. Extensive experiments across benchmarks spanning diverse data scales and modalities demonstrate that AdaS-ViT consistently achieves highly competitive performance. Notably, on ImageNet-1k it consumes only 1.02 to 2.45 millijoules per image and attains an accuracy-per-energy ratio of 32$\times$ to 72$\times$, operating in an economical inference regime and thereby pushing the energy-efficiency boundary further.

## Implementing Binary Transformers with Approximate Memories with Permutation Invariance Enabled Selective (PIES) Protection

DevFeed: [Implementing Binary Transformers with Approximate Memories with Permutation Invariance Enabled Selective (PIES) Protection](<https://devfeed.tech/articles/implementing-binary-transformers-with-approximate-memories-with-permutation-invariance-enabled-selective-pies-protection-142645.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.23358>)

Author: Zhen Gao, Xiao Yin, Kexin Liang, Pedro Reviriego, Shanshan Liu, Fabrizio Lombardi

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [quantization](<https://devfeed.tech/topics/quantization.md>), [Transformers](<https://devfeed.tech/topics/transformers.md>), [bert](<https://devfeed.tech/topics/bert.md>), [ecc ram](<https://devfeed.tech/topics/ecc-ram.md>)

Tags: [bert](<https://devfeed.tech/tags/bert.md>), [complexity](<https://devfeed.tech/tags/complexity.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [devices](<https://devfeed.tech/tags/devices.md>), [edge-devices](<https://devfeed.tech/tags/edge-devices.md>), [error](<https://devfeed.tech/tags/error.md>), [error-correction](<https://devfeed.tech/tags/error-correction.md>), [invariance](<https://devfeed.tech/tags/invariance.md>), [memory](<https://devfeed.tech/tags/memory.md>), [model](<https://devfeed.tech/tags/model.md>), [overhead](<https://devfeed.tech/tags/overhead.md>), [performance](<https://devfeed.tech/tags/performance.md>), [permutation](<https://devfeed.tech/tags/permutation.md>), [permutations](<https://devfeed.tech/tags/permutations.md>), [power](<https://devfeed.tech/tags/power.md>), [protection](<https://devfeed.tech/tags/protection.md>), [quantization](<https://devfeed.tech/tags/quantization.md>), [recover](<https://devfeed.tech/tags/recover.md>), [scaling](<https://devfeed.tech/tags/scaling.md>), [transformers](<https://devfeed.tech/tags/transformers.md>)

### AI overview

The study evaluates how memory bit errors affect binary Transformers, using BERT as a case study. It reports large accuracy losses only at high bit error rates, such as 5% or 10%. The proposed PIES scheme uses Transformers' invariance to row and column permutations to identify critical weights and selectively protect them. Simulations indicate that PIES recovers most performance loss at error rates up to 10%, with substantially lower overhead than traditional error correction schemes. These findings suggest binary Transformers may be feasible with approximate memory and aggressive voltage scaling on low-power edge devices.

### Source excerpt

arXiv:2611.23358v1 Announce Type: cross Abstract: Quantization is an effective technique to reduce memory storage, computation complexity and power for Transformers, and the binarization of the weights is important for meeting these requirements. Approximate memory can further reduce the power for memory, but it also introduces errors. Current research for binary Transformers focuses on task performance, but the model dependable operation against errors on the memory has not been well studied. In this paper, we first evaluate the dependability of binary Transformers with the Bidirectional Encoder Representations from Transformers (BERT) as a case study. A large accuracy loss occurs only at high Bit Error Rates (e.g. BER = 5% or 10%). Then we propose a novel Permutation Invariance Enabled Selective (PIES) scheme to protect the binary weights at very low overhead. This scheme exploits the invariance of Transformers to permutations in the rows and columns to group the critical weights and perform selective protection on those weights. Simulation results show that PIES can recover most of the performance loss caused by bit errors at BERs as high as 10%, at an overhead significantly lower than traditional error correction schemes. These results confirm the feasibility of running binary Transformers using approximate memory with aggressive voltage scaling on resource-limited low power edge devices. The code for PIES is released on github (https://github.com/YangZhutian/binary-transformer-prefix-repair).

## TraXion: Rethinking Pre-training Frameworks for Mobility and Beyond

DevFeed: [TraXion: Rethinking Pre-training Frameworks for Mobility and Beyond](<https://devfeed.tech/articles/traxion-rethinking-pre-training-frameworks-for-mobility-and-beyond-143253.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2605.06906>)

Author: Shang-Ling Hsu, Mark Tenzer, Cyrus Shahabi, Khurram Shafique

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Transformers](<https://devfeed.tech/topics/transformers.md>)

Tags: [anomaly-detection](<https://devfeed.tech/tags/anomaly-detection.md>), [authentication](<https://devfeed.tech/tags/authentication.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [electronic-health-records](<https://devfeed.tech/tags/electronic-health-records.md>), [event-streams](<https://devfeed.tech/tags/event-streams.md>), [frameworks](<https://devfeed.tech/tags/frameworks.md>), [health](<https://devfeed.tech/tags/health.md>), [mobility](<https://devfeed.tech/tags/mobility.md>)

### AI overview

TraXion is designed for multi-entity spatiotemporal event streams, where events combine location, time, and activity, entities retain signatures across trajectories, and events from different entities can be related through shared places or contexts. It uses objectives and an architecture built around these three properties rather than importing language-modeling recipes that treat trajectories as sentences and visits as tokens. The abstract reports that one checkpoint per dataset outperformed task-specific baselines on every reported location-ranking and detection metric across six public mobility datasets and four tasks. Applied unchanged, the recipe matched or exceeded prior work on enterprise authentication logs and ICU mortality prediction. These results suggest a shared framework across the evaluated mobility, security, and healthcare tasks; the excerpt does not provide metric values or methodological detail.

### Source excerpt

arXiv:2605.06906v2 Announce Type: replace Abstract: Human mobility differs from text and from generic time series in three structural ways: visits are tuple-valued events whose meaning depends on the joint distribution over location, time, and activity; users carry persistent signatures across trajectories; and visits are not independent across users, since co-location at shared places is a primary signal. Existing pre-training recipes for mobility import objectives from language modeling, treating trajectories as sentences and visits as tokens, an analogy that fails against each of the three properties above. These properties define a broader class, multi-entity spatiotemporal event streams (MESES), spanning enterprise authentication logs, electronic health records, and other event-stream domains where entities share infrastructure, schedules, or contexts. We make the properties precise as three axioms that any pre-training framework for MESES should satisfy, and introduce TraXion, whose objectives and architecture are jointly designed to meet them. A single TraXion checkpoint per dataset leads task-specific baselines on every location-ranking and detection metric across six public mobility datasets covering anomaly detection, next-POI recommendation, next-visit prediction, and social-link prediction. The same recipe, applied unchanged to enterprise authentication logs and ICU mortality prediction, matches or exceeds prior work on both, showing that event streams from domains as different as mobility, security, and healthcare can be modeled under a single framework.

## FairPGT: Partitioned Graph Transformers for Dual-Level Fair Graph Clustering

DevFeed: [FairPGT: Partitioned Graph Transformers for Dual-Level Fair Graph Clustering](<https://devfeed.tech/articles/fairpgt-partitioned-graph-transformers-for-dual-level-fair-graph-clustering-142683.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.23631>)

Author: Siamak Ghodsi, Amjad Seyedi, Sahar Tahmasebi, Eirini Ntoutsi, S\\"oren Auer

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [clustering](<https://devfeed.tech/topics/clustering.md>), [Transformers](<https://devfeed.tech/topics/transformers.md>), [graph theory](<https://devfeed.tech/topics/graph-theory.md>)

Tags: [attention](<https://devfeed.tech/tags/attention.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [bias](<https://devfeed.tech/tags/bias.md>), [clustering](<https://devfeed.tech/tags/clustering.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [distillation](<https://devfeed.tech/tags/distillation.md>), [fairness](<https://devfeed.tech/tags/fairness.md>), [graph](<https://devfeed.tech/tags/graph.md>), [information](<https://devfeed.tech/tags/information.md>), [parity](<https://devfeed.tech/tags/parity.md>), [partitions](<https://devfeed.tech/tags/partitions.md>), [ratio](<https://devfeed.tech/tags/ratio.md>), [routing](<https://devfeed.tech/tags/routing.md>), [scalability](<https://devfeed.tech/tags/scalability.md>), [transformers](<https://devfeed.tech/tags/transformers.md>)

### AI overview

FairPGT combines partitioned graph attention with two fairness controls for graph clustering. It limits attention to METIS partitions linked by routing tokens, reducing quadratic attention cost by the number of partitions. A mutual-information term targets demographic parity in cluster assignments, while nCGAR-based self-distillation guides attention toward group neutrality; the authors prove a neutrality correction bound and a condition against overshooting. Across seven benchmarks, it reports the best clustering performance and statistical parity on the three largest graphs, with a statistical-parity gap about ten times lower than DMoN, and is Pareto-dominated on one dataset. In controlled stochastic block models with aligned communities and protected groups, it reaches 97% of a derived NMI upper bound for exactly fair clustering. The abstract also reports that either fairness control alone can leave a different failure: assignment fairness can make attention group-heterophilous, while attention fairness alone does not guarantee fair clusters.

### Source excerpt

arXiv:2611.23631v1 Announce Type: cross Abstract: Clustering on attributed graphs can produce demographically biased clusters, particularly when communities align with protected groups. Graph transformers can amplify this bias through same-group attention, while global attention limits scalability. We propose FairPGT, a partitioned graph transformer for fair graph clustering. FairPGT restricts attention to METIS partitions and connects them through routing tokens, which divides the quadratic cost of attention by the number of partitions. A graph-aware self-training target guides clustering, with a provably bounded deviation from the standard DEC target. Fairness is imposed at two levels: a mutual-information term promotes demographic parity in cluster assignments, while a normalized cross-group attention ratio (nCGAR) guides attention toward group neutrality through self-distillation. We prove that the nCGAR correction moves attention toward neutrality and establish a condition preventing overshooting. Across seven benchmarks, FairPGT achieves the best clustering performance and statistical parity on the three largest graphs (Credit, Pokec-n, Pokec-z), with a statistical-parity gap approximately ten times lower than DMoN, and is Pareto-dominated on only one dataset. On controlled stochastic block models, FairPGT attains 97% of the derived upper bound on NMI for exactly fair clustering when communities and protected groups coincide. These graphs further show that, under strong alignment, assignment-level fairness alone makes the attention group-heterophilous, whereas attention-level fairness alone improves neutrality without ensuring fair clusters. Code for reproducibility is available at https://github.com/SiamakGhodsi/FairPGT.

## Learning through Internalization

DevFeed: [Learning through Internalization](<https://devfeed.tech/articles/learning-through-internalization-143278.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2606.20937>)

Author: Nikolaos Tsilivis, Nirmit Joshi, Marko Medvedev, Julia Kempe, Nathan Srebro

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Neural Network](<https://devfeed.tech/topics/neural-network.md>), [Transformers](<https://devfeed.tech/topics/transformers.md>)

Tags: [analysis](<https://devfeed.tech/tags/analysis.md>), [autoregressive](<https://devfeed.tech/tags/autoregressive.md>), [chain-of-thought](<https://devfeed.tech/tags/chain-of-thought.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [distribution](<https://devfeed.tech/tags/distribution.md>), [learning](<https://devfeed.tech/tags/learning.md>), [processes](<https://devfeed.tech/tags/processes.md>), [simulation](<https://devfeed.tech/tags/simulation.md>), [supervision](<https://devfeed.tech/tags/supervision.md>), [systems](<https://devfeed.tech/tags/systems.md>), [transformer](<https://devfeed.tech/tags/transformer.md>), [transformers](<https://devfeed.tech/tags/transformers.md>)

### AI overview

The paper studies how neural networks absorb explicit procedures into their weights. In transformer simulations of semiautomata, internalizing chain-of-thought tokens is associated with progressively worse out-of-distribution performance, and some semiautomata are harder to internalize than others. For parity learning, the authors prove that a simplified one-layer transformer can first learn with explicit chain-of-thought supervision, then learn to compute parity directly as those tokens are progressively removed. They state that learning parity from data without this supervision is computationally hard, and relate internalization to Positive Distribution Shift.

### Source excerpt

arXiv:2606.20937v2 Announce Type: replace Abstract: We study internalization processes, by which neural-network-based systems absorb an explicit computational procedure into their own weights, and how they facilitate learning. We investigate how transformers internalize the simulation of semiautomata by internalizing chain-of-thought (CoT) tokens, which classes of semiautomata are harder to internalize, and expose the flip side of internalization, that is, a progressive degradation of out-of-distribution performance. We then provide the first provable analysis of successful internalization: for the task of learning parities, we show that a simplified one-layer transformer provably first learns the target with explicit CoT supervision and then internalizes the autoregressive generation as CoT tokens are progressively removed, learning to directly compute the parity. This task is computationally hard to learn from data without CoT supervision. Finally, we discuss how learning through internalization relates to the \textit{Positive Distribution Shift} phenomenon recently introduced by~\citet{Med+26}.

## Exact Sequence Interpolation with Transformers

DevFeed: [Exact Sequence Interpolation with Transformers](<https://devfeed.tech/articles/exact-sequence-interpolation-with-transformers-141517.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2502.02270>)

Author: Albert Alcalde, Giovanni Fantuzzi, Enrique Zuazua

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Transformers](<https://devfeed.tech/topics/transformers.md>), [Sequences](<https://devfeed.tech/topics/sequences.md>)

Tags: [attention](<https://devfeed.tech/tags/attention.md>), [blocks](<https://devfeed.tech/tags/blocks.md>), [clustering](<https://devfeed.tech/tags/clustering.md>), [complexity](<https://devfeed.tech/tags/complexity.md>), [convergence](<https://devfeed.tech/tags/convergence.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [dataset](<https://devfeed.tech/tags/dataset.md>), [global](<https://devfeed.tech/tags/global.md>), [learning](<https://devfeed.tech/tags/learning.md>), [low-rank](<https://devfeed.tech/tags/low-rank.md>), [math-oc](<https://devfeed.tech/tags/math-oc.md>), [models](<https://devfeed.tech/tags/models.md>), [performance](<https://devfeed.tech/tags/performance.md>), [self-attention](<https://devfeed.tech/tags/self-attention.md>), [sequence](<https://devfeed.tech/tags/sequence.md>), [softmax](<https://devfeed.tech/tags/softmax.md>), [stat-ml](<https://devfeed.tech/tags/stat-ml.md>), [theoretical](<https://devfeed.tech/tags/theoretical.md>), [training](<https://devfeed.tech/tags/training.md>), [transformers](<https://devfeed.tech/tags/transformers.md>)

### AI overview

The paper constructs transformers that exactly interpolate datasets of finite input sequences in ℝ^d (d >= 2) when each output sequence is no longer than its input. For N examples with output lengths m^j, the construction uses O(Σ_j m^j) blocks and O(d Σ_j m^j) parameters, with bounds independent of input sequence length. It alternates feed-forward and self-attention layers, uses self-attention's clustering effect, and employs low-rank attention matrices. The authors first establish the result for hardmax attention, then extend it to softmax. They also report convergence guarantees to a global minimizer for learning problems under regularized training strategies.

### Source excerpt

arXiv:2502.02270v4 Announce Type: replace Abstract: We prove that transformers can exactly interpolate datasets of finite input sequences in $\mathbb{R}^d$, $d\geq 2$, with corresponding output sequences of smaller or equal length. Specifically, given $N$ sequences of arbitrary but finite lengths in $\mathbb{R}^d$ and output sequences of lengths $m^1, \dots, m^N \in \mathbb{N}$, we construct a transformer with $\mathcal{O}(\sum_{j=1}^N m^j)$ blocks and $\smash{\mathcal{O}(d \sum_{j=1}^N m^j)}$ parameters that exactly interpolates the dataset. Our construction provides complexity estimates that are independent of the input sequence length, by alternating feed-forward and self-attention layers and by capitalizing on the clustering effect inherent to the latter. Our novel constructive method also uses low-rank parameter matrices in the self-attention mechanism, a common feature of practical transformer implementations. These results are first established in the hardmax self-attention setting, where the geometric structure permits an explicit and quantitative analysis, and are then extended to the softmax setting. Finally, we demonstrate the applicability of our exact interpolation construction to learning problems, in particular by providing convergence guarantees to a global minimizer under regularized training strategies. Our analysis contributes to the theoretical understanding of transformer models, offering an explanation for their excellent performance in exact sequence-to-sequence interpolation tasks.

## Multimodal Masked Diffusion Autoencoder for Modeling Impaired Driving

DevFeed: [Multimodal Masked Diffusion Autoencoder for Modeling Impaired Driving](<https://devfeed.tech/articles/multimodal-masked-diffusion-autoencoder-for-modeling-impaired-driving-141861.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.23211>)

Author: John Gideon, Anshul Gupta, Megan Applegate-Kenton, Kayli Battel, Todd J Rowell, Patricio Reyes Gomez, Laporsha Dees, Emily Sumner, Avinash Balachandran, Guy Rosman

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Transformers](<https://devfeed.tech/topics/transformers.md>), [diffusion-transformers](<https://devfeed.tech/topics/diffusion-transformers.md>), [autonomous vehicles](<https://devfeed.tech/topics/autonomous-vehicles.md>), [Image](<https://devfeed.tech/topics/image.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [attention](<https://devfeed.tech/tags/attention.md>), [autoencoder](<https://devfeed.tech/tags/autoencoder.md>), [behavior](<https://devfeed.tech/tags/behavior.md>), [code](<https://devfeed.tech/tags/code.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-ro](<https://devfeed.tech/tags/cs-ro.md>), [data](<https://devfeed.tech/tags/data.md>), [dataset](<https://devfeed.tech/tags/dataset.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [decoding](<https://devfeed.tech/tags/decoding.md>), [design](<https://devfeed.tech/tags/design.md>), [detection](<https://devfeed.tech/tags/detection.md>), [diffusion](<https://devfeed.tech/tags/diffusion.md>), [driving](<https://devfeed.tech/tags/driving.md>), [effects](<https://devfeed.tech/tags/effects.md>), [environment](<https://devfeed.tech/tags/environment.md>), [improvements](<https://devfeed.tech/tags/improvements.md>), [inference](<https://devfeed.tech/tags/inference.md>), [input](<https://devfeed.tech/tags/input.md>), [modeling](<https://devfeed.tech/tags/modeling.md>), [multimodal](<https://devfeed.tech/tags/multimodal.md>)

### AI overview

ID-MAE combines impairment-label prediction with self-supervised reconstruction of masked gaze and trajectory, aiming to improve impairment inference when cognitive-state labels are costly. The authors introduce DIVERT, a driving-simulator dataset with 103 hours from 109 participants, covering ten cognitive, visual, and emotional impairment inductions and recording gaze, vehicle state, and scene video. On location-matched DIVERT pairs, ID-MAE reaches a paired AUC of 0.706 for impairment detection, versus 0.590 without reconstruction loss; the abstract also reports improvements on existing datasets and strong trajectory and gaze prediction. The authors report that visually demanding distractions are separable from cognitive distractions. Code and data are planned for release upon acceptance.

### Source excerpt

arXiv:2611.23211v1 Announce Type: cross Abstract: Human cognitive state (e.g., distraction, stress), gaze, and behavior interact with the driving environment in subtle ways that AI could use to assist drivers, but ground-truth labels for human cognitive state levels are costly to obtain at scale. We propose a multimodal masked-diffusion autoencoder, ID-MAE, that adds a self-supervised objective to impairment prediction: alongside predicting the impairment label as a masked token, it reconstructs masked gaze and trajectory. To probe the effects of different modalities and types of impairments on model inference, we introduce DIVERT, a driving-simulator dataset containing 103 hours of driving data (n = 109) with ten impairment inductions spanning cognitive, visual, and emotional states, recorded with gaze, vehicle state, and scene video. We demonstrate how ID-MAE affords significant improvements in impairment inference results on both the DIVERT dataset and existing datasets, and enables strong results in additional tasks, such as trajectory and gaze prediction. For location-matched pairs, our model achieves a paired AUC of 0.706 for impairment detection on DIVERT, compared to 0.590 without reconstruction loss, improving over existing baselines. We probe the design space in terms of input modalities, masking scheme, positional encoding, attention mechanism, and decoding. We also investigate the interaction between various impairments in training and inference, demonstrating that visually demanding distractions are separable from cognitive distractions. Code and data will be available upon acceptance.

## FrameRouter: Frame Budget Routing for Long-Video Understanding on Video-MME-v2 and Beyond

DevFeed: [FrameRouter: Frame Budget Routing for Long-Video Understanding on Video-MME-v2 and Beyond](<https://devfeed.tech/articles/framerouter-frame-budget-routing-for-long-video-understanding-on-video-mme-v2-and-beyond-140111.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.23055>)

Author: Bingjun Luo, Jialin Guo, Siqi Li

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Transformers](<https://devfeed.tech/topics/transformers.md>)

Tags: [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [budget](<https://devfeed.tech/tags/budget.md>), [code](<https://devfeed.tech/tags/code.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [demand](<https://devfeed.tech/tags/demand.md>), [frame](<https://devfeed.tech/tags/frame.md>), [inference](<https://devfeed.tech/tags/inference.md>), [policies](<https://devfeed.tech/tags/policies.md>), [query](<https://devfeed.tech/tags/query.md>), [question](<https://devfeed.tech/tags/question.md>), [router](<https://devfeed.tech/tags/router.md>), [routing](<https://devfeed.tech/tags/routing.md>), [sampling](<https://devfeed.tech/tags/sampling.md>), [test](<https://devfeed.tech/tags/test.md>), [v2](<https://devfeed.tech/tags/v2.md>), [video](<https://devfeed.tech/tags/video.md>)

### AI overview

FrameRouter allocates different frame budgets to long-video questions while keeping the average budget equal to a uniform reference. The training-free test-time framework uses an evidence-demand router, a budget-constrained allocation rule, and a frozen bank of sampling policies; it changes how much visual evidence each question receives while leaving the samplers unchanged. On Video-MME-v2, different question subsets preferred different fixed policies, and uniformly increasing frame counts did not reliably improve every metric. The authors report that routing improves question answering across Video-MME-v2 and several other long-video benchmarks using the same average visual budget. The supplied text gives no benchmark-specific scores or further implementation details.

### Source excerpt

arXiv:2611.23055v1 Announce Type: new Abstract: Long-video question answering benchmarks like Video-MME-v2 typically evaluate systems under a uniform frame budget for every question, despite stark variation in the evidence each query actually demands. We analyze this mismatch on Video-MME-v2 and find that different question subsets prefer different frozen frame-budget policies, while uniformly increasing the frame count does not reliably improve all metrics. These findings motivate fixed-mean frame-budget routing, a controlled inference setting in which each question may receive a different frame budget while the average budget matches a uniform reference. We introduce FrameRouter, a training-free test-time framework that combines an evidence-demand router, a budget-constrained allocation rule, and a frozen bank of frame-sampling policies. By controlling how much visual evidence each question receives while leaving the underlying samplers unchanged, FrameRouter is orthogonal to query-aware frame selection. Across Video-MME-v2 and several long-video understanding benchmarks, FrameRouter improves long-video question answering by using the same average visual budget more selectively across questions. Code is available at https://github.com/bingjunluo/FrameRouter.

## As Student Rollouts Continue, Visual Backdoors Keep Growing

DevFeed: [As Student Rollouts Continue, Visual Backdoors Keep Growing](<https://devfeed.tech/articles/as-student-rollouts-continue-visual-backdoors-keep-growing-140030.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.22446>)

Author: Zane Fu, Yanzhao Zheng, Jinze Liu, Ziting Qin, Qianshan Wei, Jihuai Zhu, Yipeng Yu, Baohua Dong, Hangcheng Zhu, Ruohui Huang

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [LLM security](<https://devfeed.tech/topics/llm-security.md>), [post-training](<https://devfeed.tech/topics/post-training.md>), [Transformers](<https://devfeed.tech/topics/transformers.md>)

Tags: [attack](<https://devfeed.tech/tags/attack.md>), [backdoors](<https://devfeed.tech/tags/backdoors.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [distillation](<https://devfeed.tech/tags/distillation.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [post-training](<https://devfeed.tech/tags/post-training.md>), [probability](<https://devfeed.tech/tags/probability.md>), [responses](<https://devfeed.tech/tags/responses.md>), [security](<https://devfeed.tech/tags/security.md>), [student](<https://devfeed.tech/tags/student.md>), [supervision](<https://devfeed.tech/tags/supervision.md>), [theoretical](<https://devfeed.tech/tags/theoretical.md>), [trigger](<https://devfeed.tech/tags/trigger.md>), [zero](<https://devfeed.tech/tags/zero.md>)

### AI overview

The abstract reports that visual backdoors can propagate through on-policy distillation (OPD) even without teacher-generated malicious responses or explicit trigger injection. Across multiple model families and datasets, backdoors rarely emerge at low poisoning ratios, but attack success rapidly approaches 100% after a critical threshold is crossed. The authors attribute propagation to the cumulative anomalous probability mass assigned by the teacher to student-generated response trajectories, and propose a trigger design that enables propagation with zero explicit trigger injection. The source contrasts this threshold-like behavior with the more gradual propagation seen in conventional offline distillation; it does not specify the threshold or provide defense guidance.

### Source excerpt

arXiv:2611.22446v1 Announce Type: cross Abstract: Conventional knowledge distillation can facilitate the propagation of backdoors from teacher models to student models. However, how this risk manifests in On-Policy Distillation (OPD) remains insufficiently studied. OPD has recently emerged as an important post-training paradigm for Vision-Language Models (VLMs). Unlike traditional offline distillation, OPD does not require teacher-generated responses. Instead, it uses the teacher's probability distributions to supervise response trajectories generated online by the student. Since the initial student model typically generates benign responses and receives no explicit supervision from malicious target responses, OPD appears to offer some inherent resistance to backdoor propagation. In this paper, we challenge this intuition and reveal a more subtle risk of visual backdoor propagation in OPD. Through experiments across multiple model families and datasets, together with theoretical analysis, we identify three main findings. First, unlike the relatively gradual propagation observed in traditional offline distillation, backdoors rarely emerge in OPD at low poisoning ratios. However, the attack success rate rapidly approaches 100% once a critical threshold is exceeded. Second, backdoor propagation does not depend on explicit malicious responses. Instead, it is driven by the continuous accumulation of anomalous probability mass assigned by the teacher. Finally, based on these findings, we propose a novel trigger design that enables visual backdoor propagation with zero explicit trigger injection. These findings further reveal an overlooked mechanism of backdoor propagation in OPD and highlight its potential security risks.

## LoopPrune: Recurrent Pruning for Looped Diffusion Transformers

DevFeed: [LoopPrune: Recurrent Pruning for Looped Diffusion Transformers](<https://devfeed.tech/articles/loopprune-recurrent-pruning-for-looped-diffusion-transformers-141806.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.22757>)

Author: Weizhong Huang, Huaijia Kou, Yan Zhang, Fei Chao, Xiawu Zheng, Liujuan Cao

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [diffusion-transformers](<https://devfeed.tech/topics/diffusion-transformers.md>), [Post-training optimization](<https://devfeed.tech/topics/post-training-optimization.md>), [Transformers](<https://devfeed.tech/topics/transformers.md>)

Tags: [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [diffusion](<https://devfeed.tech/tags/diffusion.md>), [diffusion-transformers](<https://devfeed.tech/tags/diffusion-transformers.md>), [pruning](<https://devfeed.tech/tags/pruning.md>), [recurrent](<https://devfeed.tech/tags/recurrent.md>), [transformers](<https://devfeed.tech/tags/transformers.md>)

### AI overview

LoopPrune prunes shared structures in looped diffusion Transformers by estimating recurrent importance from shared-removal effects and position variation, then reconstructing weights from pooled recurrent input statistics. During recovery, it matches complete denoising predictions and text guidance at identical student states, updating the student against a fixed dense teacher. The abstract reports that Looped-DiT B/16 retains 93.14% of dense average accuracy across six benchmarks at 30% structured parameter pruning, 9.49 percentage points above the best baseline. For pruned SDXSv3, results across PyTorch, vLLM-Omni and SGLang Diffusion show 34.36-44.64% lower peak allocated GPU memory and 1.46-1.74x faster denoising than dense models. These are abstract-level claims; further experimental details are not provided.

### Source excerpt

arXiv:2611.22757v1 Announce Type: new Abstract: Looped diffusion Transformers reuse shared blocks to deepen computation and match larger models with fewer parameters, reducing storage and deployment burdens. Yet repeated attention and MLP computations remain costly. Pruning reduces storage and computation by removing redundant parameters or units. Training-free methods struggle to resolve recurrent contributions, while training-based methods lack explicit constraints on position-specific errors affecting later inputs and final outputs. We propose LoopPrune, a pruning and recovery framework addressing these requirements. Recurrent importance estimation combines shared-removal effects and position variation to select a shared structure; shared reconstruction pools recurrent input statistics to compensate weights. Loop-position recovery compares complete denoising predictions at identical student states and matches text guidance, updating all student parameters against a fixed dense teacher. Extensive experiments across four open-source models, six benchmarks, and three pruning paradigms demonstrate effectiveness. At 30% structured parameter pruning, Looped-DiT B/16 retains 93.14% of dense six-benchmark average accuracy, exceeding the best baseline by 9.49%. Across PyTorch, vLLM-Omni, and SGLang Diffusion, pruned SDXSv3 reduces peak allocated GPU memory by 34.36-44.64% and accelerates denoising 1.46-1.74x over dense models.

## Every Measurement, Every Direction, All at Once: Multimodal Flow Matching for Molecules and Spectra

DevFeed: [Every Measurement, Every Direction, All at Once: Multimodal Flow Matching for Molecules and Spectra](<https://devfeed.tech/articles/every-measurement-every-direction-all-at-once-multimodal-flow-matching-for-molecules-and-spectra-141733.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.22098>)

Author: Thorben Prein, Elton Pan, Rafael G\\'omez-Bombarelli, Santiago Miret

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Transformers](<https://devfeed.tech/topics/transformers.md>), [Neural Network](<https://devfeed.tech/topics/neural-network.md>)

Tags: [3d](<https://devfeed.tech/tags/3d.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [denoising](<https://devfeed.tech/tags/denoising.md>), [experiment](<https://devfeed.tech/tags/experiment.md>), [flow-matching](<https://devfeed.tech/tags/flow-matching.md>), [generation](<https://devfeed.tech/tags/generation.md>), [generative](<https://devfeed.tech/tags/generative.md>), [generative-models](<https://devfeed.tech/tags/generative-models.md>), [graph](<https://devfeed.tech/tags/graph.md>), [ir](<https://devfeed.tech/tags/ir.md>), [masking](<https://devfeed.tech/tags/masking.md>), [measurements](<https://devfeed.tech/tags/measurements.md>), [model](<https://devfeed.tech/tags/model.md>), [models](<https://devfeed.tech/tags/models.md>), [multimodal](<https://devfeed.tech/tags/multimodal.md>), [physics-chem-ph](<https://devfeed.tech/tags/physics-chem-ph.md>), [prediction](<https://devfeed.tech/tags/prediction.md>), [random](<https://devfeed.tech/tags/random.md>), [signal](<https://devfeed.tech/tags/signal.md>), [structure](<https://devfeed.tech/tags/structure.md>), [supervision](<https://devfeed.tech/tags/supervision.md>), [tasks](<https://devfeed.tech/tags/tasks.md>)

### AI overview

MOSAIC uses random masking to turn each experiment's measurements into multiple conditioning and prediction tasks, training one flow matching transformer for any-to-any generation across molecular 3D structures and UV, IR, and Raman spectra. On QM9S, it reports 88.8% molecular graph recovery, 23.2% better structure generation than the strongest spectra-to-structure baseline, and stronger spectral generation than a structure-to-spectra specialist. Ablations report improved structure generation over structure-only training across model sizes. The model also accepts arbitrary subsets of available measurements and supports molecular identification by matching generated spectra to experimental references.

### Source excerpt

arXiv:2611.22098v1 Announce Type: new Abstract: Generative modeling in science is often data-limited, where experiments are expensive. However, each experiment may yield several coupled measurements of the same physical entity. Standard generative models often train only on a single direction, discarding supervision from the rest. We argue that each experiment contains more training signal than this asymmetry allows the model to exploit: random masking enables measurements to serve as either conditioning inputs or prediction targets, yielding multiple supervised denoising tasks from a single experiment. We investigate this premise with MOSAIC, a multimodal flow matching transformer over molecular 3D structure and UV, IR, and Raman spectra. MOSAIC is a single model that is capable of any-to-any generation: generating any subset of modalities from any disjoint subset of the others. On QM9S, MOSAIC achieves 88.8% molecular graph recovery, outperforming the strongest spectra-to-structure baseline by 23.2% in structure generation, while simultaneously outperforming a structure-to-spectra specialist model in spectral generation. Ablations show that the any-to-any objective improves structure generation over structure-only training across model sizes. Furthermore, MOSAIC supports arbitrary subsets of available measurements and enables molecular identification by matching generated spectra to experimental references.

[Next page](<https://devfeed.tech/topics/transformers.md?cursor=WyIyMDI2LTExLTE4VDA1OjAwOjAwKzAwOjAwIiwgIjVhY2VjYmVlLTgxNjQtNGZkYy05MWM1LTk3NzFjNTZhYThiYSJd>)