# LLM Techniques

A technical topic covering techniques for training, optimizing, and serving large language models.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## PosteriorBench: Evaluating Behavior Posterior Sampling in Language Models

DevFeed: [PosteriorBench: Evaluating Behavior Posterior Sampling in Language Models](<https://devfeed.tech/articles/posteriorbench-evaluating-behavior-posterior-sampling-in-language-models-145096.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.25735>)

Author: Zhiyang Xun, Daniel D. Johnson, Sarah Schwettmann, Neil Chowdhury

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [LLM evaluation / benchmarking](<https://devfeed.tech/topics/llm-evaluation-benchmarking.md>), [LLM Techniques](<https://devfeed.tech/topics/llm-techniques.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [qwen](<https://devfeed.tech/topics/qwen.md>)

Tags: [algorithm](<https://devfeed.tech/tags/algorithm.md>), [algorithms](<https://devfeed.tech/tags/algorithms.md>), [behavior](<https://devfeed.tech/tags/behavior.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [compute](<https://devfeed.tech/tags/compute.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [data](<https://devfeed.tech/tags/data.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [kl-divergence](<https://devfeed.tech/tags/kl-divergence.md>), [language](<https://devfeed.tech/tags/language.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [prompting](<https://devfeed.tech/tags/prompting.md>), [qwen3-5](<https://devfeed.tech/tags/qwen3-5.md>), [responses](<https://devfeed.tech/tags/responses.md>), [sampling](<https://devfeed.tech/tags/sampling.md>), [tasks](<https://devfeed.tech/tags/tasks.md>), [training](<https://devfeed.tech/tags/training.md>), [transcripts](<https://devfeed.tech/tags/transcripts.md>)

### AI overview

PosteriorBench evaluates methods for sampling language model transcripts conditioned on rare behaviors, measuring compute cost and fidelity to the true posterior with forward and reverse KL divergence. Its 21 tasks use Qwen3.5-0.8B and cover behavior probabilities from 10⁻⁵ to 10⁻¹, with exact posterior samples drawn from a corpus of one billion labeled responses. In the evaluated methods, prompting is cheap but remains far from the posterior, while fine-tuning on posterior samples is often too costly; several prior training methods do no better. No evaluated method combines efficiency and fidelity, leaving cheap, faithful posterior sampling an open problem.

### Source excerpt

arXiv:2611.25735v1 Announce Type: new Abstract: Rare language model behaviors can be difficult to study because obtaining them through ordinary sampling can require a very large number of samples. This motivates the behavior posterior sampling problem: given a language model and a behavior of interest, is there an algorithm that efficiently samples from the model's distribution over transcripts that exhibit this behavior? We introduce PosteriorBench, a benchmark for comparing algorithms by their compute cost and by how close their samples are to the true posterior in forward and reverse KL divergence. PosteriorBench contains 21 tasks on Qwen3.5-0.8B with behavior probabilities from $10^{-5}$ to $10^{-1}$, each with exact posterior samples from a corpus of one billion labeled responses. We find that no method we evaluate is both efficient and faithful: simple methods like prompting are cheap but plateau at a high KL divergence from the posterior, fine-tuning on posterior samples is often cost-prohibitive, and several training methods from prior work do not perform better. Building a posterior sampler that can cheaply approximate the true posterior remains an open problem, and we hope PosteriorBench can measure progress towards this objective. Code and data are available at https://github.com/TransluceAI/PosteriorBench.

## Interference Bounds and Merge-Aware Training Dynamics for Composable Fine-Tuned Models

DevFeed: [Interference Bounds and Merge-Aware Training Dynamics for Composable Fine-Tuned Models](<https://devfeed.tech/articles/interference-bounds-and-merge-aware-training-dynamics-for-composable-fine-tuned-models-145056.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.25430>)

Author: Alireza Moayedikia

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [AI Foundation Models](<https://devfeed.tech/topics/ai-foundation-models.md>), [LLM Techniques](<https://devfeed.tech/topics/llm-techniques.md>), [Enhancing AI model performance](<https://devfeed.tech/topics/enhancing-ai-model-performance.md>)

Tags: [alignment](<https://devfeed.tech/tags/alignment.md>), [composability](<https://devfeed.tech/tags/composability.md>), [control](<https://devfeed.tech/tags/control.md>), [cross](<https://devfeed.tech/tags/cross.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [fine](<https://devfeed.tech/tags/fine.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [functional](<https://devfeed.tech/tags/functional.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [math-oc](<https://devfeed.tech/tags/math-oc.md>), [merge](<https://devfeed.tech/tags/merge.md>), [metric](<https://devfeed.tech/tags/metric.md>), [model](<https://devfeed.tech/tags/model.md>), [models](<https://devfeed.tech/tags/models.md>), [rank](<https://devfeed.tech/tags/rank.md>), [study](<https://devfeed.tech/tags/study.md>), [training](<https://devfeed.tech/tags/training.md>), [transformer](<https://devfeed.tech/tags/transformer.md>)

### AI overview

The paper bounds the excess loss from merging fine-tuned task updates using an interference functional that captures cross-task curvature overlap and signed update alignment. In the quadratic regime, this functional equals excess loss and measures alignment in the curvature metric. A curvature-guided regularizer bounds displacement in the base curvature metric during training; under a Loewner comparison with task curvatures, it also bounds the curvature component of interference inversely with penalty strength. It does not similarly reduce the first-order term, which arises because regularized solutions are not stationary for the task loss. Experiments confirm the bounds in an exact non-quadratic model, where the first-order term dominates under strong penalties and the curvature-guided penalty performs slightly better than an isotropic one. In a three-seed transformer study, interference is rank-correlated with merge degradation, but no more than Euclidean displacement.

### Source excerpt

arXiv:2611.25430v1 Announce Type: new Abstract: Combining fine-tuned models by composing their parameter updates has become a standard alternative to multi-task training. We formalize composability as a property of a collection of task vectors and prove that the excess loss of the merged model is controlled by an interference functional of the cross-task curvature overlap and the signed alignment of the updates. In the quadratic regime this functional equals the excess loss, giving a criterion that is necessary as well as sufficient and that measures the angle between task vectors in the curvature metric rather than the Euclidean one. Fine-tuning under a curvature-guided regularizer -- constructed from curvature statistics of the kind memory-efficient optimizers maintain -- keeps displacement in the base curvature metric below a ceiling set by the penalty coefficient at every iterate, at the standard $O(1/\sqrt{T})$ non-convex rate. Under a single Loewner comparison between the base statistic and the task curvatures, this control converts into a ceiling on the interference functional, inversely proportional to the penalty coefficient, so that the curvature term of the bound is controlled during training. A first-order term, present because regularized solutions are not stationary for the task loss, is not reduced in the same way. Experiments on an exact non-quadratic model confirm the budget and the ceiling, show the first-order term dominating for strong penalties, and find the curvature-guided penalty slightly better than an isotropic one. In a three-seed transformer study the functional is rank-correlated with merging degradation, but no more strongly than the Euclidean displacement norm.

## Beam Model Technical Report

DevFeed: [Beam Model Technical Report](<https://devfeed.tech/articles/beam-model-technical-report-145663.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.26358>)

Author: Reflection Team, Adam Abdulhamid, Osama Abuelsorour, Yasmin Alameddine, Carlo Alfano, Duarte Alves, Brandon Amos, Selvaraj Anandaraj, Arseni Anisimovich, George Antoniadis, Ioannis Antonoglou, Sheila

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [LLM Techniques](<https://devfeed.tech/topics/llm-techniques.md>), [Mixture of Experts (MoE)](<https://devfeed.tech/topics/mixture-of-experts-moe.md>), [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [context window](<https://devfeed.tech/topics/context-window.md>), [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>), [post-training](<https://devfeed.tech/topics/post-training.md>), [quantization](<https://devfeed.tech/topics/quantization.md>), [distributed-training](<https://devfeed.tech/topics/distributed-training.md>), [Model Optimizer](<https://devfeed.tech/topics/model-optimizer.md>), [NVFP4](<https://devfeed.tech/topics/nvfp4.md>)

Tags: [agentic-workloads](<https://devfeed.tech/tags/agentic-workloads.md>), [asynchronous](<https://devfeed.tech/tags/asynchronous.md>), [beam](<https://devfeed.tech/tags/beam.md>), [co-design](<https://devfeed.tech/tags/co-design.md>), [coding](<https://devfeed.tech/tags/coding.md>), [context-window](<https://devfeed.tech/tags/context-window.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [fp8](<https://devfeed.tech/tags/fp8.md>), [gb300](<https://devfeed.tech/tags/gb300.md>), [mixture-of-experts-moe](<https://devfeed.tech/tags/mixture-of-experts-moe.md>), [nvfp4](<https://devfeed.tech/tags/nvfp4.md>), [open](<https://devfeed.tech/tags/open.md>), [reasoning](<https://devfeed.tech/tags/reasoning.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [speculative-decoding](<https://devfeed.tech/tags/speculative-decoding.md>), [throughput](<https://devfeed.tech/tags/throughput.md>)

### AI overview

Beam is a 501.3B-parameter sparse MoE model with 23.4B active parameters, designed for coding, reasoning, and agentic workloads. The abstract reports a 1M-token context window enabled by two-stage midtraining on verifiable, correctness-anchored data, following pretraining on 23.8T tokens. Post-training used asynchronous RL on 10.5K GB300 GPUs, producing over 100M rollouts in four weeks; measured capabilities continued to improve as RL compute increased, with no plateau reported. Hardware-model co-design, FP8/NVFP4 quantization, and a 1.93B-parameter DFlash 2 speculative decoding drafter target serving throughput. The report claims 3-4x less inference compute than many comparable models. On its safety evaluation, Beam refused or pivoted on 88.9% of harmful requests while complying with 81.5% of harmless requests; the text says its harmful-request refusal/pivot rate was highest among open-weight models on the Pareto frontier.

### Source excerpt

arXiv:2611.26358v1 Announce Type: cross Abstract: Beam is a sparse Mixture-of-Experts (MoE) model with 501.3B total parameters and 23.4B active parameters, specifically built and optimized for coding, reasoning, and agentic workloads. Beam delivers frontier level reasoning, while requiring 3-4x less inference compute than many models of its class. Pretrained on 23.8T diverse, curated, high-quality tokens, Beam features a 1M-token context window achieved through a novel two-stage midtraining phase, built on verifiable, correctness-anchored data. Post-training leveraged a large-scale, fully asynchronous reinforcement learning (RL) campaign across 10.5K GB300 GPUs, generating over 100M rollouts over 4 weeks of training. We believe this is one of the largest scale RL runs conducted by any open lab to date. Across our evaluation suite, capabilities continued to improve as we increased RL compute, with no sign of a plateau. Supported by deep hardware-model co-design, FP8/NVFP4 quantization, and a 1.93B-parameter speculative decoding drafter (DFlash 2), Beam maximizes serving throughput with efficient, flexible deployment. Averaged across conversation turns and harm categories, Beam refuses or pivots on 88.9% of harmful requests while complying with 81.5% of harmless requests, achieving the highest refusal/pivot rate on harmful requests among open-weight models on the Pareto frontier.

## Gradients for Interventions and Activation Values for Detection: Targeted Feature Learning in Language Models

DevFeed: [Gradients for Interventions and Activation Values for Detection: Targeted Feature Learning in Language Models](<https://devfeed.tech/articles/gradients-for-interventions-and-activation-values-for-detection-targeted-feature-learning-in-language-models-144935.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2609.32355>)

Author: Jonathan Drechsel, Steffen Herbold

Published: 2026-11-20T05:00:00Z

Content type: comparison

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Model Development](<https://devfeed.tech/topics/model-development.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [LLM Techniques](<https://devfeed.tech/topics/llm-techniques.md>)

Tags: [activation](<https://devfeed.tech/tags/activation.md>), [autoencoders](<https://devfeed.tech/tags/autoencoders.md>), [compare](<https://devfeed.tech/tags/compare.md>), [concepts](<https://devfeed.tech/tags/concepts.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [detection](<https://devfeed.tech/tags/detection.md>), [feature](<https://devfeed.tech/tags/feature.md>), [gradient](<https://devfeed.tech/tags/gradient.md>), [interpretability](<https://devfeed.tech/tags/interpretability.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [learning](<https://devfeed.tech/tags/learning.md>), [models](<https://devfeed.tech/tags/models.md>), [performance](<https://devfeed.tech/tags/performance.md>), [results](<https://devfeed.tech/tags/results.md>), [signal](<https://devfeed.tech/tags/signal.md>), [steering](<https://devfeed.tech/tags/steering.md>), [study](<https://devfeed.tech/tags/study.md>), [tasks](<https://devfeed.tech/tags/tasks.md>), [values](<https://devfeed.tech/tags/values.md>)

### AI overview

The abstract reports that targeted feature quality depends on both the model signal and estimator, and that detection and intervention measure complementary properties. In a comparison across 15 tasks and ten language models, contrastive activation value methods performed strongest for detection, while gradient-based methods--especially contrastive means--performed strongest for intervention. Within the contrastive mean methods, activation gradients substantially improved intervention over activation values without requiring parameter-space intervention. The study compares six targeted methods spanning activation values, activation gradients, and parameter gradients, paired with either a contrastive mean or a learned one-dimensional encoder-decoder, against pretrained Sparse Autoencoders.

### Source excerpt

arXiv:2609.32355v2 Announce Type: replace-cross Abstract: Model-internal features can be studied through both their ability to identify a specified concept and their causal effect when manipulated, e.g., through steering or weight editing. A prominent approach to feature learning is Sparse Autoencoders (SAEs), which learn broad feature dictionaries whose relation to particular concepts is typically identified post hoc. However, many interpretability questions are instead hypothesis-driven and concern a concept specified in advance. We study this setting as targeted feature learning, where a single feature is constructed for such a predefined concept. We present a controlled comparison across three model signals (activation values, activation gradients, and parameter gradients) and two estimators (contrastive mean and a learned one-dimensional encoder-decoder), yielding six targeted methods, with CAA and GRADIEND as existing instances and four new methods covering the remaining combinations. We compare these methods against pretrained SAEs across 15 tasks and ten language models, evaluating both detection and causal intervention. Across models, the strongest detection performance is achieved by contrastive activation value methods, whereas the strongest intervention performance is achieved by gradient signals, most strongly with a contrastive mean. Notably, within the contrastive mean family, activation gradients substantially improve intervention over activation values without requiring parameter space interventions. Overall, our results show that targeted feature quality depends jointly on the model signal and estimator, with detection and intervention capturing complementary properties.

## Closing Performance Gap for Diffusion Language Models under Low Inference Budgets

DevFeed: [Closing Performance Gap for Diffusion Language Models under Low Inference Budgets](<https://devfeed.tech/articles/closing-performance-gap-for-diffusion-language-models-under-low-inference-budgets-145100.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.25763>)

Author: Hao Wu, Sipeng Chen, Jingwen Fu, Yuwang Wang

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [LLM Techniques](<https://devfeed.tech/topics/llm-techniques.md>), [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>), [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>)

Tags: [autoregressive](<https://devfeed.tech/tags/autoregressive.md>), [budget](<https://devfeed.tech/tags/budget.md>), [code-generation](<https://devfeed.tech/tags/code-generation.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [diffusion](<https://devfeed.tech/tags/diffusion.md>), [inference](<https://devfeed.tech/tags/inference.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [mathematical-reasoning](<https://devfeed.tech/tags/mathematical-reasoning.md>), [mbpp](<https://devfeed.tech/tags/mbpp.md>), [recovery](<https://devfeed.tech/tags/recovery.md>), [stat-ml](<https://devfeed.tech/tags/stat-ml.md>), [training](<https://devfeed.tech/tags/training.md>)

### AI overview

CaRR improves submission quality for editable diffusion language models under tight inference budgets without adding backbone forwards. Its Token Commitment Remasking (TCR) remasks selected native pure replacements and refills them on the next scheduled edit using updated context; Submission Commitment Recovery (SCR) uses the refinement trajectory to recover supported candidates when the final response fails task-specific output requirements. Both work within the existing mask/edit schedule. Across four budgets, experiments in code generation and mathematical reasoning report consistent gains in average accuracy over native ME-DLM, largest at tight budgets. At one-eighth of the full scheduled forward budget, CaRR raises six-metric average accuracy by 7.44 percentage points, with gains of 11.37 points on MBPP and 10.05 on MBPP+.

### Source excerpt

arXiv:2611.25763v1 Announce Type: cross Abstract: Diffusion language models offer an architectural route to low-budget generation by updating multiple tokens in parallel, which can reduce the number of sequential decoding steps relative to token-by-token autoregressive decoding. However, realizing this advantage at low budgets is challenging, as generation quality often declines when the number of denoising and refinement steps is reduced. Editable diffusion models address this quality loss through post-draft refinement, but useful intermediate results may still be lost before final submission. We introduce CaRR (Commitment-aware Refinement and Recovery), a training-free framework which improves submission quality from a fixed editable checkpoint through updated-context refill and trajectory-based recovery within its existing forward schedule. Token Commitment Remasking (TCR) remasks selected native pure replacements and refills them on the next scheduled edit forward using the updated context. Submission Commitment Recovery (SCR) uses the resulting refinement trajectory to recover supported candidates when the final response does not satisfy task-specific output requirements. Both modules operate within the native mask/edit schedule without additional backbone forwards. Experiments on code generation and mathematical reasoning show consistent gains in average accuracy over native ME-DLM across four inference budgets, with the largest gains under tight budgets. At one-eighth of the full scheduled forward budget, CaRR improves the six-metric average accuracy by 7.44 pp, including gains of 11.37 and 10.05 on MBPP and MBPP+.

## Separating Interpretability and Reconstruction in Sparse Autoencoders

DevFeed: [Separating Interpretability and Reconstruction in Sparse Autoencoders](<https://devfeed.tech/articles/separating-interpretability-and-reconstruction-in-sparse-autoencoders-145016.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.25165>)

Author: Anh Tien Nguyen, Dimitris Samaras, Mahdi S. Hosseini

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Representation learning](<https://devfeed.tech/topics/representation-learning.md>), [LLM Techniques](<https://devfeed.tech/topics/llm-techniques.md>)

Tags: [autoencoders](<https://devfeed.tech/tags/autoencoders.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [interpretability](<https://devfeed.tech/tags/interpretability.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [mechanistic-interpretability](<https://devfeed.tech/tags/mechanistic-interpretability.md>), [representation-learning](<https://devfeed.tech/tags/representation-learning.md>), [topic-modeling](<https://devfeed.tech/tags/topic-modeling.md>)

### AI overview

DualSAE separates sparse feature interpretability from reconstruction fidelity by adding a Residual Branch to a standard SAE. The Sparse Branch learns a sparse code for dominant semantics; the Residual Branch models omitted non-semantic information. A decorrelation loss limits overlap between their reconstruction contributions, and two-stage training stabilizes sparse learning before coupling the branches. Across encoder-SAE configurations on two vision datasets and two language corpora, the abstract reports mean relative gains of about 29% in reconstruction fidelity and 34% in monosemanticity score, plus consistent gains in topic modeling, hypothesis generation, and LLM steering. These are reported results from the described evaluations; the excerpt provides no detailed metrics or setup.

### Source excerpt

arXiv:2611.25165v1 Announce Type: new Abstract: Sparse Autoencoders (SAEs) are effective tools for mechanistic interpretability by decomposing neural representations into sparse and interpretable features. However, a single sparse representation in traditional SAEs is responsible for both reconstruction fidelity and interpretability, which are not guaranteed to align. To separate these objectives, we propose DualSAE, a modular framework that augments standard SAEs with a plug-in Residual Branch. A standard SAE serves as the Sparse Branch, learning a sparse code that contains the dominant semantics of the input. The added Residual Branch models the non-semantic information omitted by the Sparse Branch. We ensure that the contributions of the two branches towards the overall reconstruction have minimal overlap through a decorrelation loss. We also adopt a two-stage training strategy that first stabilizes sparse representation learning before coupling with the Residual Branch. Across diverse encoder-SAE configurations on two vision datasets and two language corpora, DualSAE achieves mean relative improvements of approximately 29% in reconstruction fidelity and 34% in interpretability (measured by monosemanticity score). Moreover, with more interpretable representations, DualSAE yields consistent performance gains in the downstream applications of topic modeling, hypothesis generation, and LLM steering. Qualitative analysis further suggests that the Sparse Branch indeed preserves dominant semantic content, whereas the Residual Branch captures non-semantic information to improve reconstruction fidelity.

## Relax Within, Balance Across: Load Balancing under Composition Shifts in Vision-Language Mixture-of-Experts

DevFeed: [Relax Within, Balance Across: Load Balancing under Composition Shifts in Vision-Language Mixture-of-Experts](<https://devfeed.tech/articles/relax-within-balance-across-load-balancing-under-composition-shifts-in-vision-language-mixture-of-experts-145359.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2608.00574>)

Author: Ziang Wu, Peng Jin, Qishen Yin, Munan Ning, Hao Li, Peizhen Zhang, Huihui Zhou, Li Yuan

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Mixture of Experts (MoE)](<https://devfeed.tech/topics/mixture-of-experts-moe.md>), [vlm](<https://devfeed.tech/topics/vlm.md>), [post-training](<https://devfeed.tech/topics/post-training.md>), [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>), [LLM Techniques](<https://devfeed.tech/topics/llm-techniques.md>)

Tags: [balance](<https://devfeed.tech/tags/balance.md>), [balancing](<https://devfeed.tech/tags/balancing.md>), [cancellation](<https://devfeed.tech/tags/cancellation.md>), [changes](<https://devfeed.tech/tags/changes.md>), [code](<https://devfeed.tech/tags/code.md>), [composition](<https://devfeed.tech/tags/composition.md>), [counts](<https://devfeed.tech/tags/counts.md>), [cross](<https://devfeed.tech/tags/cross.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [errors](<https://devfeed.tech/tags/errors.md>), [experts](<https://devfeed.tech/tags/experts.md>), [fast](<https://devfeed.tech/tags/fast.md>), [fixed](<https://devfeed.tech/tags/fixed.md>), [flat](<https://devfeed.tech/tags/flat.md>), [gap](<https://devfeed.tech/tags/gap.md>), [image](<https://devfeed.tech/tags/image.md>), [language](<https://devfeed.tech/tags/language.md>), [load](<https://devfeed.tech/tags/load.md>), [load-balancing](<https://devfeed.tech/tags/load-balancing.md>), [mixture-of-experts](<https://devfeed.tech/tags/mixture-of-experts.md>), [vision](<https://devfeed.tech/tags/vision.md>)

### AI overview

On a fixed vision-language MoE router, balancing image and text tokens in one pooled auxiliary loss can conceal opposing modality load errors: they cancel near the training image-to-text ratio, but load skews as resolution, tiling, or image count changes that mix. The paper derives a quadratic response law under fixed modality profiles, with the image-text load gap governing sensitivity. It also reports that patches from an image route similarly as patch counts vary, allowing large images to dominate flat pooling. ReBA separates modality losses by token fraction and weights images equally, allowing per-image specialization while balancing visual load overall. Across four split backbones, the authors report stable balance across resolutions, tilings, and image counts without hurting accuracy; large modality gaps in released native MoEs suggest potential for native post-training.

### Source excerpt

arXiv:2608.00574v2 Announce Type: replace-cross Abstract: Vision-language mixture-of-experts (MoE) models such as Qwen3-VL need balanced loads for fast parallel serving. Standard training pools image and text tokens into one auxiliary loss, assuming training balance holds at inference. Yet we find a surprising mismatch: on a fixed router, balance shifts sharply whenever resolution, dynamic tiling, or image count changes the image-to-text token ratio. Splitting load by modality reveals cross-modal load cancellation: opposing image and text errors cancel near the training ratio, hiding strong skew until the mix changes. To quantify this shift, we derive an exact quadratic law: under fixed modality profiles, one reference run predicts the full response curve, and the image-text gap governs sensitivity. Meanwhile, patches within an image route similarly while patch counts vary, letting large images dominate flat pooling. We propose ReBA (Relax Within, Balance Across) to separate modality losses by token fraction and weight images equally, letting each image specialize while keeping overall visual load even. Across four split backbones, ReBA turns fragile single-ratio balance into stable balance across resolutions, tilings, and image counts without hurting accuracy. Large modality gaps in released native MoEs also show its promise for native post-training. Code is available at https://github.com/ZiangWu-77/ReBA.

## TideCache: Efficient Hybrid LLM Serving with Transparent Cache Management and Layer-Wise Streaming

DevFeed: [TideCache: Efficient Hybrid LLM Serving with Transparent Cache Management and Layer-Wise Streaming](<https://devfeed.tech/articles/tidecache-efficient-hybrid-llm-serving-with-transparent-cache-management-and-layer-wise-streaming-145202.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.26619>)

Author: Jinbin Luo, Xiaozhe Yao, Shang Yang, Zhijian Liu, Song Han, Qinghao Hu

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Distributed Computing](<https://devfeed.tech/sources/arxiv-distributed-computing.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>), [LLM Techniques](<https://devfeed.tech/topics/llm-techniques.md>), [Cache](<https://devfeed.tech/topics/cache.md>), [sglang](<https://devfeed.tech/topics/sglang.md>), [memory pool](<https://devfeed.tech/topics/memory-pool.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [architectures](<https://devfeed.tech/tags/architectures.md>), [attention](<https://devfeed.tech/tags/attention.md>), [backend](<https://devfeed.tech/tags/backend.md>), [bounded](<https://devfeed.tech/tags/bounded.md>), [cache](<https://devfeed.tech/tags/cache.md>), [capacity](<https://devfeed.tech/tags/capacity.md>), [components](<https://devfeed.tech/tags/components.md>), [concurrency](<https://devfeed.tech/tags/concurrency.md>), [context](<https://devfeed.tech/tags/context.md>), [cpu](<https://devfeed.tech/tags/cpu.md>), [cs-dc](<https://devfeed.tech/tags/cs-dc.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [execution](<https://devfeed.tech/tags/execution.md>), [hybrid](<https://devfeed.tech/tags/hybrid.md>), [layer](<https://devfeed.tech/tags/layer.md>), [llm](<https://devfeed.tech/tags/llm.md>), [llm-serving](<https://devfeed.tech/tags/llm-serving.md>), [management](<https://devfeed.tech/tags/management.md>), [serving](<https://devfeed.tech/tags/serving.md>), [streaming](<https://devfeed.tech/tags/streaming.md>), [transparent](<https://devfeed.tech/tags/transparent.md>)

### AI overview

TideCache serves hybrid LLMs by coordinating GPU and host memory for global-attention and bounded-state caches. Its Transparent Cache Pool reallocates physical pages while preserving virtual addresses and backend interfaces; its Cache Streamer uses bounded-state computation intervals to prefetch global-attention prefixes into shared GPU slots, overlapping transfers with computation while keeping generated KV resident. A Request Scheduler chooses resident or streamed execution and adjusts capacity based on request size and queue state. Implemented in SGLang, TideCache was evaluated on GB300 and GH200 across seven model/platform configurations. Compared with SGLang using HiCache, it completed up to 2.04x as many agent turns, reduced overall P50 per-token latency by 20-59%, and improved agent P50 TTFT by 1.40-3.05x.

### Source excerpt

arXiv:2611.26619v1 Announce Type: cross Abstract: Hybrid architectures have become mainstream among open-weight language models. Their global-attention and bounded-state layers maintain heterogeneous caches whose demands vary with context length and concurrency. We present TideCache, a serving system on CPU--GPU superchips with three components for allocating GPU memory among caches and managing their residency across GPU and host memory. The Transparent Cache Pool redistributes physical pages among cache components behind fixed virtual addresses, preserving cache layouts and backend interfaces. The Cache Streamer uses computation intervals provided by bounded-state layers to prefetch the next global-attention layer's prefix into GPU slots shared across global-attention layers, reducing GPU residency while overlapping layer-wise transfers with computation. Generated KV remains resident. The Request Scheduler selects resident or streamed execution and triggers capacity adjustments according to request size and queue state. Together, they let HBM released by streaming accommodate other cache components and additional requests. Implemented in SGLang and evaluated on GB300 and GH200 across seven model/platform configurations, TideCache completes up to 2.04x as many agent turns, reduces overall P50 per-token latency by 20--59%, and improves agent P50 TTFT by 1.40--3.05x compared with SGLang with HiCache.

## DTS: Delta-Based Tile Scoring for Efficient Long-Context Sparse Attention

DevFeed: [DTS: Delta-Based Tile Scoring for Efficient Long-Context Sparse Attention](<https://devfeed.tech/articles/dts-delta-based-tile-scoring-for-efficient-long-context-sparse-attention-145120.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.25938>)

Author: Emre U\\u{g}ur, Jiawen Qi, Qinyu Chen

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>), [LLM Techniques](<https://devfeed.tech/topics/llm-techniques.md>), [Transformer](<https://devfeed.tech/topics/transformer.md>), [context window](<https://devfeed.tech/topics/context-window.md>), [PyTorch](<https://devfeed.tech/topics/pytorch.md>)

Tags: [adaptive](<https://devfeed.tech/tags/adaptive.md>), [anchor](<https://devfeed.tech/tags/anchor.md>), [attention](<https://devfeed.tech/tags/attention.md>), [bottleneck](<https://devfeed.tech/tags/bottleneck.md>), [bounded](<https://devfeed.tech/tags/bounded.md>), [concurrently](<https://devfeed.tech/tags/concurrently.md>), [content](<https://devfeed.tech/tags/content.md>), [context](<https://devfeed.tech/tags/context.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [delta](<https://devfeed.tech/tags/delta.md>), [flashattention](<https://devfeed.tech/tags/flashattention.md>), [free](<https://devfeed.tech/tags/free.md>), [full](<https://devfeed.tech/tags/full.md>), [gpus](<https://devfeed.tech/tags/gpus.md>), [h100](<https://devfeed.tech/tags/h100.md>), [language](<https://devfeed.tech/tags/language.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [large](<https://devfeed.tech/tags/large.md>), [large-language-models-llms](<https://devfeed.tech/tags/large-language-models-llms.md>), [long-context](<https://devfeed.tech/tags/long-context.md>), [sparse-attention](<https://devfeed.tech/tags/sparse-attention.md>)

### AI overview

DTS reduces the cost of scoring blocks for sparse long-context attention by choosing query and key anchors based on local vector changes: it uses fewer anchors in redundant regions and more in heterogeneous ones. Parallel delta packing speeds selection by scanning tile-aligned chunks concurrently; selected tiles are computed with the original query, key, and value vectors. The method is training-free. On LLaMA-3.1-8B-Instruct, average RULER and LongBench scores were within 0.10 percentage points of full attention, with RULER tested up to 128K tokens. At 128K, scoring-pipeline latency was 38% lower than XAttention; at 512K, prefill was 3.3x faster than a PyTorch full-attention baseline using FlashAttention-2 on four NVIDIA H100 GPUs.

### Source excerpt

arXiv:2611.25938v1 Announce Type: new Abstract: Large language models (LLMs) increasingly need to process long documents and codebases, making long-context attention a major prefill bottleneck. Block sparse attention reduces computation by skipping unimportant blocks of the attention matrix, but scoring blocks accurately can itself be expensive. We propose Delta-Based Tile Scoring (DTS), a training-free, content-adaptive tile scoring method for sparse block attention. DTS selects query and key anchors according to local vector changes, using fewer representatives in redundant regions while retaining more detail in heterogeneous regions. To accelerate anchor selection, parallel delta packing partitions the query and key sequences into tile-aligned chunks and performs bounded local scans over these chunks concurrently. The selected tiles are then computed using the original query, key, and value vectors. On LLaMA-3.1-8B-Instruct, DTS keeps average RULER and LongBench scores within 0.10 percentage points of full attention, with RULER evaluated up to 128K tokens. At 128K tokens, DTS reduces measured scoring-pipeline latency by 38% compared with XAttention. On four NVIDIA H100 GPUs, it achieves a 3.3$\times$ prefill speedup over a PyTorch full-attention baseline using FlashAttention-2 at 512K tokens.

## Periodic and Harmonic Periodic RoPE for Length Extrapolation in LLMs

DevFeed: [Periodic and Harmonic Periodic RoPE for Length Extrapolation in LLMs](<https://devfeed.tech/articles/periodic-and-harmonic-periodic-rope-for-length-extrapolation-in-llms-145339.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2605.27980>)

Author: Simin Huo, Ning Li

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Large Language Model](<https://devfeed.tech/topics/llm.md>), [long-context](<https://devfeed.tech/topics/long-context.md>), [LLM Techniques](<https://devfeed.tech/topics/llm-techniques.md>)

Tags: [attention](<https://devfeed.tech/tags/attention.md>), [code](<https://devfeed.tech/tags/code.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [embedding](<https://devfeed.tech/tags/embedding.md>), [error](<https://devfeed.tech/tags/error.md>), [extrapolation](<https://devfeed.tech/tags/extrapolation.md>), [hp](<https://devfeed.tech/tags/hp.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [llms](<https://devfeed.tech/tags/llms.md>), [positional-encoding](<https://devfeed.tech/tags/positional-encoding.md>), [results](<https://devfeed.tech/tags/results.md>), [rope](<https://devfeed.tech/tags/rope.md>), [rotation](<https://devfeed.tech/tags/rotation.md>), [sliding-window](<https://devfeed.tech/tags/sliding-window.md>), [tasks](<https://devfeed.tech/tags/tasks.md>), [test](<https://devfeed.tech/tags/test.md>), [token](<https://devfeed.tech/tags/token.md>), [training](<https://devfeed.tech/tags/training.md>), [window](<https://devfeed.tech/tags/window.md>), [wrap](<https://devfeed.tech/tags/wrap.md>), [yarn](<https://devfeed.tech/tags/yarn.md>)

### AI overview

Periodic RoPE (P-RoPE) and Harmonic Periodic RoPE (HP-RoPE) are designed to extend context length alongside sliding window attention. P-RoPE repeats the first W RoPE angles, limiting the model to W distinct angles, but creates a wrap error at window boundaries. HP-RoPE selects frequencies that complete whole turns within W tokens to remove that error. In comparisons where models differed only in window-layer position encoding, both methods kept perplexity nearly flat from the 768-token training length to 16k; standard RoPE, NTK, and YaRN showed rising perplexity. At 16k, P-RoPE and HP-RoPE scored 99-100% on a format-free Passkey test, compared with 11-37% for the RoPE interpolations.

### Source excerpt

arXiv:2605.27980v2 Announce Type: replace-cross Abstract: The ability to process long contexts is crucial for large language models (LLMs) to perform long-horizon tasks. Rotary position embedding (RoPE) gives every token a new rotation angle, so a model eventually meets angles it never saw in training, i.e., position exhaustion. We propose Periodic RoPE (P-RoPE), a positional encoding mechanism that operates in conjunction with sliding window attention. It repeats the first sliding-window-size \(W\) angles of the RoPE table, so the model needs only \(W\) distinct angles. P-RoPE works but introduces a wrap error at the window boundary. We further propose Harmonic Periodic RoPE (HP-RoPE), which removes the error by choosing frequencies that complete whole turns in \(W\) tokens. We compare models that share the same architecture, data, and schedule and differ only in the window-layer position encoding: standard RoPE, P-RoPE, and HP-RoPE. The results show that P-RoPE and HP-RoPE keep perplexity nearly flat from the training length \(768\) to \(16\)k, while RoPE, NTK, and YaRN rise. In a format-free Passkey test at \(16\)k, they remain at \(99\)--\(100\%\), against \(11\)--\(37\%\) for those RoPE interpolations. The code is available at \href{https://github.com/Cominder/miniwin}{https://github.com/Cominder/miniwin}.

## Characterizing High Bandwidth Flash for LLM Serving

DevFeed: [Characterizing High Bandwidth Flash for LLM Serving](<https://devfeed.tech/articles/characterizing-high-bandwidth-flash-for-llm-serving-144948.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2609.39131>)

Author: Zack Yu, Chloe Wong, Coleman Hooper, Minjae Lee, Wonjun Kang, Youngjin Cho, Michael W. Mahoney, Yakun Sophia Shao, Kurt Keutzer, Amir Gholami

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Distributed Computing](<https://devfeed.tech/sources/arxiv-distributed-computing.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Cache](<https://devfeed.tech/topics/cache.md>), [systems](<https://devfeed.tech/topics/systems.md>), [LLM Techniques](<https://devfeed.tech/topics/llm-techniques.md>)

Tags: [agentic-workloads](<https://devfeed.tech/tags/agentic-workloads.md>), [cs-ar](<https://devfeed.tech/tags/cs-ar.md>), [cs-dc](<https://devfeed.tech/tags/cs-dc.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [efficiency](<https://devfeed.tech/tags/efficiency.md>), [endurance](<https://devfeed.tech/tags/endurance.md>), [hbm](<https://devfeed.tech/tags/hbm.md>), [high-bandwidth-flash](<https://devfeed.tech/tags/high-bandwidth-flash.md>), [kv-cache](<https://devfeed.tech/tags/kv-cache.md>), [llm](<https://devfeed.tech/tags/llm.md>), [llm-serving](<https://devfeed.tech/tags/llm-serving.md>), [scheduling](<https://devfeed.tech/tags/scheduling.md>), [simulations](<https://devfeed.tech/tags/simulations.md>)

### AI overview

Trace-driven simulations of high-throughput agentic LLM serving find that adding high-bandwidth flash (HBF) to an HBM-HBF-host memory hierarchy can reduce completion time by 21.1-86.9% versus HBM-only systems in the evaluated workloads. Modeled energy savings reach 62.4%, varying with workload and weight placement. Because HBF has access costs and limited write endurance, the study pairs data placement with buffered cache-aware scheduling; in one evaluated configuration, this scheduling raises estimated HBF write lifetime from 0.99 to 13.63 years. These are simulation results, not measurements of deployed systems.

### Source excerpt

arXiv:2609.39131v3 Announce Type: replace Abstract: Large language model (LLM) serving requires substantial memory to store model weights and KV caches. Agentic workloads compound this pressure through repeated interactions over growing contexts, making it important to retain KV cache for reuse. High-bandwidth flash (HBF) offers a way to expand accelerator memory capacity for LLM serving, but its access costs and limited write endurance complicate its use. We evaluate HBF for high-throughput agentic serving across system design and scheduling choices to understand when additional capacity improves serving performance and energy efficiency. We introduce an HBM-HBF-host hierarchical storage system and buffered cache-aware scheduling, and use trace-driven simulations to analyze their effects on performance, energy consumption, and HBF write lifetime. Across the evaluated workloads, the fastest HBF-augmented systems reduce completion time by 21.1-86.9% relative to HBM-only systems. Modeled energy savings reach 62.4%, with benefits depending on the workload and weight placement. Buffered cache-aware scheduling extends estimated HBF write lifetime from 0.99 to 13.63 years in the evaluated configuration. These results demonstrate the importance of coordinating data placement and scheduling in serving LLM with HBF, positioning HBF as an actively utilized lower-level memory tier rather than passive storage.

## CHORUS: Coverage-Guided Agentic RL and Policy Consolidation for Testbench Stimulus Generation

DevFeed: [CHORUS: Coverage-Guided Agentic RL and Policy Consolidation for Testbench Stimulus Generation](<https://devfeed.tech/articles/chorus-coverage-guided-agentic-rl-and-policy-consolidation-for-testbench-stimulus-generation-144918.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2608.10090>)

Author: Hejia Zhang, Sheng Lu, Zhijing Wu, Yitong Ma, Zhongming Yu, Wentao Ni, Chia-Tung Ho, Jishen Zhao

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>), [LLM Techniques](<https://devfeed.tech/topics/llm-techniques.md>), [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>)

Tags: [agentic](<https://devfeed.tech/tags/agentic.md>), [coverage](<https://devfeed.tech/tags/coverage.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [generation](<https://devfeed.tech/tags/generation.md>), [policy](<https://devfeed.tech/tags/policy.md>), [rl](<https://devfeed.tech/tags/rl.md>), [stimulus](<https://devfeed.tech/tags/stimulus.md>), [testbench](<https://devfeed.tech/tags/testbench.md>)

### AI overview

CHORUS uses coverage-guided reinforcement learning to improve LLM-generated testbench stimulus: the model drafts testbenches, then repairs its lowest-scoring draft using simulator feedback. On CVDP-ECov, this raised LLM4Cov agentic Pass@1 from 69.2% to 85.5%. Three separately trained checkpoints performed similarly overall but succeeded on different designs; averaging their weights produced a dense 4B model at 86.7%. Selecting a teacher per prompt by simulator reward and distilling it into a 4B model reached 88.0%, reported as 13.5 percentage points above DeepSeek-R1 (671B).

### Source excerpt

arXiv:2608.10090v2 Announce Type: replace-cross Abstract: Testbenches written by large language models can compile and run while leaving entire datapaths idle. We present CHORUS, the first framework to combine coverage-guided agentic reinforcement learning (RL) with policy consolidation for testbench stimulus generation. We train LLM4Cov's final supervised checkpoint to draft testbenches and repair its lowest-scoring draft using simulator feedback, raising CVDP-ECov agentic Pass@1 from 69.2% to 85.5%. Applying the same training to three checkpoints yields similar overall pass rates, yet the policies succeed on different designs. Averaging their weights combines these strengths in a single dense 4B model that reaches 86.7% agentic Pass@1. When simulator rewards select the teacher for each training prompt, the distilled 4B model reaches 88.0%, 13.5 percentage points above DeepSeek-R1 (671B).

## Visualizing Information Entropy in Language Models

DevFeed: [Visualizing Information Entropy in Language Models](<https://devfeed.tech/articles/visualizing-information-entropy-in-language-models-143868.md>)

Original publisher: [Read original article](<https://machinelearningmastery.com/visualizing-information-entropy-in-language-models/>)

Author: Iván Palomares Carrascosa

Published: 2026-11-19T11:00:00Z

Content type: article

Language: en

Sources: [MachineLearningMastery.com](<https://devfeed.tech/sources/machinelearningmastery-com.md>)

Topics: [LLM Techniques](<https://devfeed.tech/topics/llm-techniques.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Python](<https://devfeed.tech/topics/python.md>)

Tags: [cross-entropy](<https://devfeed.tech/tags/cross-entropy.md>), [entropy](<https://devfeed.tech/tags/entropy.md>), [information](<https://devfeed.tech/tags/information.md>), [information-theory](<https://devfeed.tech/tags/information-theory.md>), [language-models](<https://devfeed.tech/tags/language-models.md>)

### AI overview

Information entropy measures uncertainty: a uniform distribution has high entropy, while a distribution concentrated on one outcome has low entropy. The excerpt connects this idea to next-token prediction, where a language model assigns probabilities across its vocabulary. During training, cross-entropy compares those predictions with a one-hot distribution for the correct token; minimizing it pushes probability toward the target. The described Python and Plotly example uses a five-word vocabulary and three hypothetical predictions. It highlights that a model can have high cross-entropy either by being uncertain or by confidently choosing the wrong word. The excerpt does not include the code or numerical results.

### Source excerpt

In this article, you will learn how information entropy and cross-entropy loss connect classical information theory to the way large language models are trained to...

## Self-Rewarded Reinforcement Learning for Better-Structured Medical Explanations in Small Language Models

DevFeed: [Self-Rewarded Reinforcement Learning for Better-Structured Medical Explanations in Small Language Models](<https://devfeed.tech/articles/self-rewarded-reinforcement-learning-for-better-structured-medical-explanations-in-small-language-models-142819.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.24650>)

Author: Prakhar Mishra, Bhavya Yajush Awasthi, Tanuja, Nishant

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>), [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [LLM Techniques](<https://devfeed.tech/topics/llm-techniques.md>), [post-training](<https://devfeed.tech/topics/post-training.md>), [lora](<https://devfeed.tech/topics/lora.md>)

Tags: [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [medical](<https://devfeed.tech/tags/medical.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [structured](<https://devfeed.tech/tags/structured.md>)

### AI overview

The study finds that a lightweight, self-rewarded reinforcement learning step can make medical explanations from Mistral-7B longer and more organized while preserving their underlying medical content. It applies PPO with LoRA to 2,000 Meerkat-derived prompts, using a reward that favors length, lexical diversity, and punctuation and penalizes very short responses. About 170 training steps on one NVIDIA A100 produced a stable increase in average reward. The authors discuss both the promise and limitations of using this approach to shape explanation style.

### Source excerpt

arXiv:2611.24650v1 Announce Type: new Abstract: Small language models (SLMs) can run on commodity hardware and support privacy-preserving, on-premises clinical deployment, but their medical explanations are often short and poorly structured compared with large language models. Recent supervised approaches such as Meerkat transfer chain-of-thought reasoning into 7B-8B backbones, yet their training objective does not directly target the style or organization of the generated rationales. Building on the instruction-tuned Mistral-7B backbone, we investigate whether a lightweight reinforcement learning (RL) step can improve explanation structure without human preference labels.We apply Proximal Policy Optimization (PPO) with LoRA adaptation to 2,000 Meerkat-derived medical prompts, guided by a simple self-supervised reward that favors longer, lexically diverse, and well-punctuated responses while penalizing very short ones. Training for about 170 PPO steps on a single NVIDIA A100 GPU yields a stable increase in the average reward. Quantitative and qualitative comparisons show that the RL-tuned model produces longer, more organized, multi-point explanations while preserving the underlying medical content.We discuss the promise and the limitations of self-rewarded RL for shaping the style of medical explanations.

## LEDGE: An Efficient Simulator for Distributed LLM Serving on Heterogeneous Edge Devices

DevFeed: [LEDGE: An Efficient Simulator for Distributed LLM Serving on Heterogeneous Edge Devices](<https://devfeed.tech/articles/ledge-an-efficient-simulator-for-distributed-llm-serving-on-heterogeneous-edge-devices-142754.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.24085>)

Author: Zihao Wang, Ye Tian, Tajana Rosing, Nalini Venkatasubramanian

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Distributed Computing](<https://devfeed.tech/sources/arxiv-distributed-computing.md>)

Topics: [LLM Techniques](<https://devfeed.tech/topics/llm-techniques.md>), [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>), [qwen](<https://devfeed.tech/topics/qwen.md>), [resource orchestration](<https://devfeed.tech/topics/resource-orchestration.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>)

Tags: [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-dc](<https://devfeed.tech/tags/cs-dc.md>), [distributed](<https://devfeed.tech/tags/distributed.md>), [edge](<https://devfeed.tech/tags/edge.md>), [edge-devices](<https://devfeed.tech/tags/edge-devices.md>), [jetson](<https://devfeed.tech/tags/jetson.md>), [latency](<https://devfeed.tech/tags/latency.md>), [llm](<https://devfeed.tech/tags/llm.md>), [llm-serving](<https://devfeed.tech/tags/llm-serving.md>), [pipeline-parallelism](<https://devfeed.tech/tags/pipeline-parallelism.md>), [qwen](<https://devfeed.tech/tags/qwen.md>), [raspberry](<https://devfeed.tech/tags/raspberry.md>), [scaling](<https://devfeed.tech/tags/scaling.md>), [simulation](<https://devfeed.tech/tags/simulation.md>), [simulator](<https://devfeed.tech/tags/simulator.md>), [source-code](<https://devfeed.tech/tags/source-code.md>)

### AI overview

LEDGE simulates distributed LLM serving on heterogeneous edge devices without executing tensor operations. It builds event graphs for non-uniform tensor and pipeline parallelism, estimates execution costs with reusable performance models calibrated from end-to-end measurements, and models shared-link contention. On a 10-device Jetson and Raspberry Pi testbed, it covered 17 model-precision combinations across four LLM families. For Qwen deployments, request-level MAPE was below 3.6%, with 135.3x aggregate speedup over physical inference. Models calibrated on one family achieved 7.44% zero-shot end-to-end latency MAPE across three others. The authors also report simulations showing effects of shared-link contention on placement and network topology on scaling; 26,773 runs across four edge environments took 83.8 minutes.

### Source excerpt

arXiv:2611.24085v1 Announce Type: cross Abstract: Evaluating distributed large language model (LLM) serving on heterogeneous edge devices requires exploring diverse model partitions, device placements, and network conditions, making repeated physical experiments costly. An efficient simulator for edge LLM serving must support diverse, fine-grained placement configurations, capture runtime behavior, and accurately predict execution times while keeping simulation overhead low. We present LEDGE, a lightweight simulator for fast, fine-grained evaluation of distributed LLM serving on heterogeneous edge devices without executing tensor operations. LEDGE constructs event graphs supporting non-uniform tensor and pipeline parallelism, estimates execution costs using reusable performance models calibrated from end-to-end measurements, and simulates shared-link contention. We evaluate LEDGE on a heterogeneous testbed of 10 Jetson and Raspberry Pi devices, covering 17 model--precision combinations across 4 LLM families. On Qwen deployments, LEDGE achieves request-level mean absolute percentage errors below 3.6% and a 135.3 times aggregate speedup over physical inference. Performance model calibrated on one LLM family yield 7.44% zero-shot end-to-end latency MAPE across three other families. Further simulations show how shared-link contention affects placement decisions and network topology constrains scaling. Across four edge environments, LEDGE completes 26,773 simulation runs in 83.8 minutes, enabling efficient exploration of deployment strategies under diverse resource conditions and serving objectives. The source code is publicly available at https://github.com/gdfwj/LEDGE.

## General progress substitutes for specific training data in language models

DevFeed: [General progress substitutes for specific training data in language models](<https://devfeed.tech/articles/general-progress-substitutes-for-specific-training-data-in-language-models-142838.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.24844>)

Author: Mark Marosi, Peter Antal

Published: 2026-11-19T05:00:00Z

Content type: news

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [LLM Techniques](<https://devfeed.tech/topics/llm-techniques.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [scaling laws](<https://devfeed.tech/topics/scaling-laws.md>)

Tags: [capabilities](<https://devfeed.tech/tags/capabilities.md>), [chess](<https://devfeed.tech/tags/chess.md>), [control](<https://devfeed.tech/tags/control.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [data](<https://devfeed.tech/tags/data.md>), [delay](<https://devfeed.tech/tags/delay.md>), [language](<https://devfeed.tech/tags/language.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [models](<https://devfeed.tech/tags/models.md>), [progress](<https://devfeed.tech/tags/progress.md>), [training](<https://devfeed.tech/tags/training.md>), [training-data](<https://devfeed.tech/tags/training-data.md>)

### AI overview

Across experiments on chess, Othello, molecular strings, bracket languages and web text, general training progress partly substituted for withheld skill-specific data. Larger models' delays on withheld skills were predicted from smaller models. A chess model trained without check-response examples ranked legal responses nearly as well as one trained on them; when withheld data was later added, models acquired the skill using a fraction of the data needed from initialization, with that fraction shrinking as training progressed. An equally difficult control rule was learned more slowly, and a programming language withheld from a 7-billion-parameter model's training text was regained similarly. The findings suggest filtering delays capabilities, but the data needed to acquire them later may decline as models improve.

### Source excerpt

arXiv:2611.24844v1 Announce Type: cross Abstract: Language models acquire their capabilities from training data, and developers withhold data during training to keep models from learning capabilities that could be misused. How long this delays a capability, and how much data is needed to acquire it later, is unknown. Here we show that the general progress a model makes on the rest of its training data substitutes for the data that was withheld. We withheld the training signal for individual skills in models of chess, Othello, molecular strings, bracket languages and web text. The delay of a withheld skill in larger models was predicted from smaller ones. A chess model never explicitly trained to respond to check ranked the legal responses almost as well as a model that was. When models were then trained on the withheld data, they acquired the skill with a small fraction of the data needed from initialization, and this fraction decreased as the models progressed. A control rule of equal difficulty was learned much more slowly. A programming language withheld from the training text of a 7-billion-parameter model was regained in the same way. Data filtering therefore delays a capability, and the data required to regain it decreases as models improve.

## NullTok: empirical-null token selection for training-free long video understanding

DevFeed: [NullTok: empirical-null token selection for training-free long video understanding](<https://devfeed.tech/articles/nulltok-empirical-null-token-selection-for-training-free-long-video-understanding-142716.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.23881>)

Author: Zhang Ziyue, Juwei Jian, Baiyang Song, Yunshan Zhong, Jun Peng, Yiyi Zhou, Rongrong Ji

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [LLM Techniques](<https://devfeed.tech/topics/llm-techniques.md>), [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>), [Language models](<https://devfeed.tech/topics/language-models.md>), [Transformers](<https://devfeed.tech/topics/transformers.md>), [qwen](<https://devfeed.tech/topics/qwen.md>)

Tags: [audio](<https://devfeed.tech/tags/audio.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [bounded](<https://devfeed.tech/tags/bounded.md>), [budget](<https://devfeed.tech/tags/budget.md>), [control](<https://devfeed.tech/tags/control.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [deduplication](<https://devfeed.tech/tags/deduplication.md>), [experimental](<https://devfeed.tech/tags/experimental.md>), [llava](<https://devfeed.tech/tags/llava.md>), [monte-carlo](<https://devfeed.tech/tags/monte-carlo.md>), [multimodal](<https://devfeed.tech/tags/multimodal.md>), [null](<https://devfeed.tech/tags/null.md>), [patch](<https://devfeed.tech/tags/patch.md>), [qwen3](<https://devfeed.tech/tags/qwen3.md>), [selection](<https://devfeed.tech/tags/selection.md>), [semantic](<https://devfeed.tech/tags/semantic.md>), [token](<https://devfeed.tech/tags/token.md>), [training](<https://devfeed.tech/tags/training.md>), [video](<https://devfeed.tech/tags/video.md>)

### AI overview

NullTok selects video tokens by calibrating each patch's query affinity against same-position scores from 64 group-matched irrelevant control queries. Under a conditional exchangeability assumption, these references yield discrete super-uniform right-tail Monte Carlo p-values. Calibrated screening and a high-recall semantic guard identify candidates, followed by visual deduplication and joint spatiotemporal budget allocation. The training-free method is evaluated with frozen LLaVA-Video-7B and Qwen3-VL-8B backbones. At a cap of 64 full-frame equivalents, without subtitles or audio, it scores 65.81% and 68.85% on Video-MME, respectively, outperforming the evaluated baselines at the same budget for both backbones.

### Source excerpt

arXiv:2611.23881v1 Announce Type: cross Abstract: Long video understanding remains a daunting challenge for existing multimodal large language models (MLLMs), mainly due to the large number of visual tokens accumulated in long videos. Currently, frame selection and token selection are two main paradigms for addressing this issue, both aiming to select relevant visual information to form a compact input for MLLMs. A natural approach combines these two paradigms by selecting frames before pruning tokens, but noisy frame-level relevance estimates can cause irreversible evidence loss. In this paper, we study this problem from the perspective of assessing the spatiotemporal distribution of query-specific evidence and propose a novel and training-free approach termed NULLTOK. For every patch across all frames, we construct an empirical-null distribution from 64 group-matched irrelevant control queries, capturing the scores expected in the absence of target-specific evidence. Target affinities are calibrated against these same-position references into right-tail Monte Carlo p-values, which are discrete super-uniform under a conditional exchangeability assumption. Calibrated screening and a high-recall semantic guard produce candidate tokens for deterministic visual deduplication and joint spatiotemporal budget allocation. To validate NullTok, we apply it to two representative MLLMs and conduct extensive experiments on a set of video benchmarks. The experimental results demonstrate its effectiveness under a bounded visual token budget. For example, it achieves 65.81% and 68.85% on Video-MME using frozen LLaVA-Video-7B and Qwen3-VL-8B backbones, respectively, at a cap of 64 full-frame equivalents (FFE), without subtitles or audio. These results outperform all evaluated baselines on Video-MME under the same budget for both backbones. Code is available at https://github.com/potato-kitty/NullTok.

## Plug-and-Play Multi-Licensee Robust Watermarking for Mixture-of-Experts LLMs

DevFeed: [Plug-and-Play Multi-Licensee Robust Watermarking for Mixture-of-Experts LLMs](<https://devfeed.tech/articles/plug-and-play-multi-licensee-robust-watermarking-for-mixture-of-experts-llms-143137.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.23838>)

Author: Jianshuo Liu, Kang G. Shin, Haining Wang

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Cryptography and Security](<https://devfeed.tech/sources/arxiv-cryptography-and-security.md>)

Topics: [Mixture of Experts (MoE)](<https://devfeed.tech/topics/mixture-of-experts-moe.md>), [LLM security](<https://devfeed.tech/topics/llm-security.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [LLM Techniques](<https://devfeed.tech/topics/llm-techniques.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>)

Tags: [adaptive](<https://devfeed.tech/tags/adaptive.md>), [attribution](<https://devfeed.tech/tags/attribution.md>), [black-box](<https://devfeed.tech/tags/black-box.md>), [cs-cr](<https://devfeed.tech/tags/cs-cr.md>), [distribution](<https://devfeed.tech/tags/distribution.md>), [embedding](<https://devfeed.tech/tags/embedding.md>), [framework](<https://devfeed.tech/tags/framework.md>), [incremental](<https://devfeed.tech/tags/incremental.md>), [intellectual-property](<https://devfeed.tech/tags/intellectual-property.md>), [large-language-models](<https://devfeed.tech/tags/large-language-models.md>), [llms](<https://devfeed.tech/tags/llms.md>), [mixture-of-experts](<https://devfeed.tech/tags/mixture-of-experts.md>), [mixture-of-experts-moe](<https://devfeed.tech/tags/mixture-of-experts-moe.md>), [model-weights](<https://devfeed.tech/tags/model-weights.md>), [modules](<https://devfeed.tech/tags/modules.md>), [moe](<https://devfeed.tech/tags/moe.md>), [robust](<https://devfeed.tech/tags/robust.md>), [robustness](<https://devfeed.tech/tags/robustness.md>), [router](<https://devfeed.tech/tags/router.md>), [routing](<https://devfeed.tech/tags/routing.md>), [watermarking](<https://devfeed.tech/tags/watermarking.md>)

### AI overview

PPMoE embeds licensee-specific watermarks in MoE LLMs by selectively merging combinations of pre-trained incremental weight modules that steer router decisions, avoiding separate per-licensee models and additional training at distribution time. For attribution, white-box analysis inspects triggered routing decisions or merged modules; black-box analysis uses output-token likelihood ratios. The abstract reports experiments across five MoE models and seven datasets, with high attribution accuracy, preserved base-model utility, and robustness to adaptive watermark-removal attacks. It does not provide quantitative results or details of the attacks.

### Source excerpt

arXiv:2611.23838v1 Announce Type: new Abstract: Mixture-of-Experts (MoE) architectures enable high-capacity large language models (LLMs) with high sparse computation, making them attractive for deployment on private hardware. However, distributing model weights exposes valuable intellectual property (IP) to leakage and unauthorized redistribution, creating an urgent need to both verify model ownership and trace leaked copies to individual licensees. Existing model-watermarking techniques are ill-suited for meeting this need because they are fragile under MoE's dynamic routing or require costly retraining to embed a distinct watermark for each licensee, thus making scalable distribution difficult. To address these challenges, we present PPMoE, the first plug-and-play multi-licensee watermarking framework for MoE LLMs. Rather than training a separate model per user, PPMoE trains a compact set of mergeable incremental weight modules that steer the MoE router toward distinct, mutually compatible routing patterns. During model distribution, the model owner selectively merges unique combinations of these pre-trained, mutually compatible modules into the base model on the fly, embedding a licensee-specific watermark without additional training. PPMoE supports reliable multi-licensee attribution under both white- and black-box threat models. In the white-box setting, attribution is done by inspecting triggered routing decisions or the merged weight modules responsible for these decisions, whereas in the black-box setting, attribution relies on analyzing output token likelihood ratios. Extensive experiments across five state-of-the-art MoE models, seven benchmark datasets, and varying number of licensees show that PPMoE achieves very high attribution accuracy while preserving base model utility and maintaining strong robustness to various adaptive watermark-removal attacks.

## A Survey on RLOps for Large Language Models

DevFeed: [A Survey on RLOps for Large Language Models](<https://devfeed.tech/articles/a-survey-on-rlops-for-large-language-models-142184.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.23357>)

Author: Lingzhe Zhang, Tong Jia, Chiming Duan, Minghua He, Leyi Pan, Kening Zheng, Zhaoyang Liu, Bolin Ding, Philip S. Yu, Ying Li

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Cryptography and Security](<https://devfeed.tech/sources/arxiv-cryptography-and-security.md>), [arXiv: Software Engineering](<https://devfeed.tech/sources/arxiv-software-engineering.md>)

Topics: [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [LLM Techniques](<https://devfeed.tech/topics/llm-techniques.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [LLM observability](<https://devfeed.tech/topics/llm-observability.md>)

Tags: [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-cr](<https://devfeed.tech/tags/cs-cr.md>), [cs-se](<https://devfeed.tech/tags/cs-se.md>), [distributed](<https://devfeed.tech/tags/distributed.md>), [efficiency](<https://devfeed.tech/tags/efficiency.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [github-repository](<https://devfeed.tech/tags/github-repository.md>), [large-language-models-llms](<https://devfeed.tech/tags/large-language-models-llms.md>), [observability](<https://devfeed.tech/tags/observability.md>), [recursive-self-improvement](<https://devfeed.tech/tags/recursive-self-improvement.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [reinforcement-learning-training](<https://devfeed.tech/tags/reinforcement-learning-training.md>), [reliability](<https://devfeed.tech/tags/reliability.md>), [shared-infrastructure](<https://devfeed.tech/tags/shared-infrastructure.md>), [survey](<https://devfeed.tech/tags/survey.md>), [taxonomy](<https://devfeed.tech/tags/taxonomy.md>)

### AI overview

The survey organizes methods for reliable and efficient LLM reinforcement-learning operations across five lifecycle stages: Training Setup, Experience Collection, Reward Computation, Policy Optimization, and Model Evaluation. It frames RLOps as a response to distributed-execution, feedback-computation, and learning-dynamics failures that can waste compute, slow training, or weaken improvements. For each stage, the survey examines operational concerns, management techniques, and evaluation evidence. It identifies future work in training observability, coordination across stages, generalizable RLOps, managing multiple runs, and supporting recursive self-improvement. The supplied text is an abstract and does not provide details on individual methods or comparative results.

### Source excerpt

arXiv:2611.23357v1 Announce Type: new Abstract: Reinforcement learning (RL) has become increasingly important for improving the reasoning, instruction-following, and task-solving capabilities of large language models (LLMs). However, scaling RL training exposes failures in distributed execution, feedback computation, and learning dynamics that can waste computational resources, slow training, and undermine model improvements. These challenges have motivated a growing body of techniques for LLM RLOps, which manages the efficiency and reliability of operations throughout the LLM reinforcement learning training lifecycle. Despite growing interest, these techniques remain scattered across research areas, with limited synthesis of their operational roles and interactions. To bridge this gap, we present a systematic survey of LLM RLOps methods. We organize existing approaches into five stages: Training Setup, Experience Collection, Reward Computation, Policy Optimization, and Model Evaluation. Within each stage, we examine operational concerns, management techniques, and evaluation evidence. Building on this taxonomy, we identify open challenges and future directions in training process observability, cross-stage coordination, generalizable RLOps, multi-run management, and RLOps for recursive self-improvement. These directions extend operational management from individual training runs to diverse training settings, shared infrastructure, and successive improvement cycles. We maintain a GitHub repository indexing relevant papers and open resources at https://github.com/zhanglingzhe0820/Awesome-RLOps.

## Beyond the Sink Token: Sink-Guided Key Editing in Vision-Language Models

DevFeed: [Beyond the Sink Token: Sink-Guided Key Editing in Vision-Language Models](<https://devfeed.tech/articles/beyond-the-sink-token-sink-guided-key-editing-in-vision-language-models-142667.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.23516>)

Author: Hui Zhu, Saleh Zare Zade, Rafi Ibn Sultan, Xiangyu Zhou, Dongxiao Zhu

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [vlm](<https://devfeed.tech/topics/vlm.md>), [LLM Techniques](<https://devfeed.tech/topics/llm-techniques.md>), [model architecture](<https://devfeed.tech/topics/model-architecture.md>), [enhancing AI model accuracy](<https://devfeed.tech/topics/enhancing-ai-model-accuracy.md>)

Tags: [attention](<https://devfeed.tech/tags/attention.md>), [attention-steering](<https://devfeed.tech/tags/attention-steering.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [boosting](<https://devfeed.tech/tags/boosting.md>), [components](<https://devfeed.tech/tags/components.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [gemma-3](<https://devfeed.tech/tags/gemma-3.md>), [generation](<https://devfeed.tech/tags/generation.md>), [image](<https://devfeed.tech/tags/image.md>), [keys](<https://devfeed.tech/tags/keys.md>), [layers](<https://devfeed.tech/tags/layers.md>), [llava](<https://devfeed.tech/tags/llava.md>), [magnitude](<https://devfeed.tech/tags/magnitude.md>), [operations](<https://devfeed.tech/tags/operations.md>), [points](<https://devfeed.tech/tags/points.md>), [position](<https://devfeed.tech/tags/position.md>), [question](<https://devfeed.tech/tags/question.md>), [reasoning](<https://devfeed.tech/tags/reasoning.md>), [reference](<https://devfeed.tech/tags/reference.md>), [steering](<https://devfeed.tech/tags/steering.md>), [vision-language-models](<https://devfeed.tech/tags/vision-language-models.md>)

### AI overview

Across four probed vision-language models, increasing positive sink-aligned key components raised attention to image and question tokens on average, while increasing negative components lowered it; the effects persisted in later layers. Using this relationship, the authors propose SHRINK, a training-free method that partially reduces sink-aligned components across visible keys during generation. On nine benchmarks with sampled reasoning, it raised mean accuracy by 5.1 percentage points for LLaVA-OneVision-7B and 2.7 points for Qwen2.5-VL-3B, with smaller gains for Qwen2.5-VL-7B and Gemma-3-4B. Rescaling these key components also replaced attention operations in GIFT and ACG with comparable accuracy on hallucination benchmarks.

### Source excerpt

arXiv:2611.23516v1 Announce Type: cross Abstract: In vision-language models (VLMs), a few sink tokens attract a disproportionate share of attention. While previous studies have examined why these tokens attract attention, we investigate how the sink-key direction affects attention to image and question tokens. Across four probed VLMs, boosting the magnitude of positive sink-aligned components increases attention on average, whereas boosting negative components decreases it; these responses persist through later layers. This identifies the native sink key as a reference for controlling attention beyond its own position. Motivated by this finding, we propose SHRINK, a training-free method that partially reduces sink-aligned components across visible keys during generation. On nine benchmarks under sampled reasoning, it improves mean accuracy by 5.1 and 2.7 percentage points on LLaVA-OneVision-7B and Qwen2.5-VL-3B, with smaller gains on the other two main models, Qwen2.5-VL-7B and Gemma-3-4B. Beyond SHRINK, rescaling sink-aligned key components can replace the attention operations of two existing attention-steering methods, GIFT and ACG, with comparable accuracy on hallucination benchmarks.

## Training Language Models for Introspective Oversight

DevFeed: [Training Language Models for Introspective Oversight](<https://devfeed.tech/articles/training-language-models-for-introspective-oversight-142648.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.23365>)

Author: Alex Lin Wang, Linus Yifeng Tang, Dami Choi, Neil Chowdhury

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [LLM Techniques](<https://devfeed.tech/topics/llm-techniques.md>), [LLM observability](<https://devfeed.tech/topics/llm-observability.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [post-training](<https://devfeed.tech/topics/post-training.md>), [Uncertainty quantification LLMs](<https://devfeed.tech/topics/uncertainty-quantification-llms.md>)

Tags: [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [deception](<https://devfeed.tech/tags/deception.md>), [evaluations](<https://devfeed.tech/tags/evaluations.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [monitoring](<https://devfeed.tech/tags/monitoring.md>), [observer](<https://devfeed.tech/tags/observer.md>), [oversight](<https://devfeed.tech/tags/oversight.md>), [training](<https://devfeed.tech/tags/training.md>), [transcript](<https://devfeed.tech/tags/transcript.md>)

### AI overview

Introspective oversight trains a separate third-person observer role to answer questions about an assistant, with answers rewarded for matching the assistant's verified behavior. The approach aims to address limits of asking the assistant to report its own misbehavior, transcript monitors' reliance on visible outputs, and activation decoders' lack of context and tendency to confabulate. On five held-out oversight evaluations, an observer trained on Qwen3.6-27B achieved mean AUROC 0.907, compared with 0.864 for honesty training, 0.828 for transcript monitoring, and 0.796 for activation oracles. Training the observer also made the assistant easier to monitor: every baseline method improved on average when used with the fine-tuned assistant. These results suggest the approach is tractable for overseeing unintended behavior.

### Source excerpt

arXiv:2611.23365v1 Announce Type: new Abstract: Language models increasingly exhibit behaviors that are difficult to detect in transcripts, such as reward hacking, deception, and sycophancy. Asking the assistant follow-up questions about its actions is unreliable, since models are generally not trained to report their own misbehavior. As a result, approaches to oversight typically involve querying external models. These face limitations: external transcript monitoring operates only on visible model outputs, and activation decoders lack full context and are prone to confabulation. We introduce introspective oversight, which trains a new role in the model to answer arbitrary questions about the assistant, rewarding answers that are verified against the assistant's actual behavior. We implement this by adding a separate, third-person observer role that answers these queries, which we can train solely for accuracy and separately from the assistant. On five held-out oversight evaluations, an observer trained on Qwen3.6-27B obtains a mean AUROC of 0.907, outperforming honesty training (0.864), transcript monitoring (0.828), and activation oracles (0.796). We further find that training an observer makes the assistant itself easier to monitor, with every baseline method improving on average when applied to the fine-tuned assistant. These results suggest that training models to answer verifiable questions about themselves is a tractable approach to overseeing unintended behavior.

## Training-Free Dynamic Upcycling of Expert Language Models

DevFeed: [Training-Free Dynamic Upcycling of Expert Language Models](<https://devfeed.tech/articles/training-free-dynamic-upcycling-of-expert-language-models-143249.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2603.29765>)

Author: Eros Fan\\\`i, O\\u{g}uzhan Ersoy

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Mixture of Experts (MoE)](<https://devfeed.tech/topics/mixture-of-experts-moe.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [LLM Techniques](<https://devfeed.tech/topics/llm-techniques.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>)

Tags: [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [domains](<https://devfeed.tech/tags/domains.md>), [experts](<https://devfeed.tech/tags/experts.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [performance](<https://devfeed.tech/tags/performance.md>), [reasoning](<https://devfeed.tech/tags/reasoning.md>), [training](<https://devfeed.tech/tags/training.md>), [upcycling](<https://devfeed.tech/tags/upcycling.md>)

### AI overview

DUME combines dense language models trained on different domains into a single Mixture of Experts model without additional training. It uses a closed-form ridge regression solution, avoiding further optimization and allowing experts to be added or removed dynamically. Across causal language modeling and reasoning experiments with models from 115M to 9B parameters, the authors report that DUME outperforms baseline approaches. In reasoning, it averages 104% of the domain expert's performance. The resulting model can also be fine-tuned to improve performance on complex reasoning tasks.

### Source excerpt

arXiv:2603.29765v2 Announce Type: replace Abstract: Large Language Models (LLMs) have achieved remarkable performance on a wide range of specialized tasks, exhibiting strong problem-solving capabilities. However, training these models is prohibitively expensive, and they often lack domain-specific expertise because they rely on general knowledge datasets. Expertise finetuning can address this issue; however, it often leads to overspecialization, and developing a single multi-domain expert remains difficult due to diverging objectives. Furthermore, multitask training is challenging due to interference and catastrophic forgetting. Existing work proposes combining the expertise of dense models within a Mixture of Experts (MoE) architecture, although this approach still requires multitask finetuning. To address these issues, we introduce Dynamic Upcycling MoE (DUME), a novel approach that reuses dense experts trained on different domains to construct a unified MoE model. Our method builds a single multitask model that preserves the capabilities of the original dense experts without requiring additional training. DUME is both cost-efficient and scalable: by leveraging the closed-form solution of ridge regression, it eliminates the need for further optimization and enables experts to be added or removed dynamically. Our extensive experiments on different tasks (in causal language modeling and reasoning settings) with several model sizes (between 115M and 9B parameters) and architectures demonstrate that DUME consistently outperforms baseline approaches. In particular, in a reasoning setting, DUME can also surpass the dense expert model's performance in its domain, achieving 104% of the dense expert's performance on average. Finally, we also show that the DUME model can be fine-tuned to further improve performance in complex reasoning scenarios. Our code is available at: github.com/gensyn-ai/dume.

## A Survey on Collaborative Mechanisms Between Large and Small Language Models

DevFeed: [A Survey on Collaborative Mechanisms Between Large and Small Language Models](<https://devfeed.tech/articles/a-survey-on-collaborative-mechanisms-between-large-and-small-language-models-142494.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2505.07460>)

Author: Yi Chen, Haiquan Wang, JiaHao Zhao, HaoHao Han

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [LLM Techniques](<https://devfeed.tech/topics/llm-techniques.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [AI Development](<https://devfeed.tech/topics/ai-development.md>), [LLM evaluation / benchmarking](<https://devfeed.tech/topics/llm-evaluation-benchmarking.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [accessible](<https://devfeed.tech/tags/accessible.md>), [adaptive](<https://devfeed.tech/tags/adaptive.md>), [advanced](<https://devfeed.tech/tags/advanced.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-applications](<https://devfeed.tech/tags/ai-applications.md>), [also](<https://devfeed.tech/tags/also.md>), [application](<https://devfeed.tech/tags/application.md>), [applications](<https://devfeed.tech/tags/applications.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [balance](<https://devfeed.tech/tags/balance.md>), [but](<https://devfeed.tech/tags/but.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [language](<https://devfeed.tech/tags/language.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [large](<https://devfeed.tech/tags/large.md>), [models](<https://devfeed.tech/tags/models.md>), [small](<https://devfeed.tech/tags/small.md>), [small-language-models](<https://devfeed.tech/tags/small-language-models.md>), [survey](<https://devfeed.tech/tags/survey.md>)

### AI overview

The survey frames collaboration between large language models (LLMs) and small language models (SLMs) as a way to balance capability against resource cost and latency: SLMs are more efficient and deployable but offer lower performance. It outlines five mechanisms--pipeline, routing, auxiliary support, distillation, and fusion--and connects them to edge uses that value low latency, privacy, personalization, or offline operation. It also identifies unresolved challenges in system overhead, inter-model consistency, task allocation, evaluation, and security and privacy. The supplied text is an abstract, so it does not provide detailed comparisons or evidence about the effectiveness of individual mechanisms.

### Source excerpt

arXiv:2505.07460v2 Announce Type: replace Abstract: Large Language Models (LLMs) deliver powerful AI capabilities but face deployment challenges due to high resource costs and latency, whereas Small Language Models (SLMs) offer efficiency and deployability at the cost of reduced performance. Collaboration between LLMs and SLMs emerges as a crucial paradigm to synergistically balance these trade-offs, enabling advanced AI applications, especially on resource-constrained edge devices. This survey provides a comprehensive overview of LLM-SLM collaboration, detailing various interaction mechanisms (pipeline, routing, auxiliary, distillation, fusion), key enabling technologies, and diverse application scenarios driven by on-device needs like low latency, privacy, personalization, and offline operation. While highlighting the significant potential for creating more efficient, adaptable, and accessible AI, we also discuss persistent challenges including system overhead, inter-model consistency, robust task allocation, evaluation complexity, and security/privacy concerns. Future directions point towards more intelligent adaptive frameworks, deeper model fusion, and expansion into multimodal and embodied AI, positioning LLM-SLM collaboration as a key driver for the next generation of practical and ubiquitous artificial intelligence.

## Harvesting Expert-Parallel Bubbles for Fine-Tuning

DevFeed: [Harvesting Expert-Parallel Bubbles for Fine-Tuning](<https://devfeed.tech/articles/harvesting-expert-parallel-bubbles-for-fine-tuning-142803.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.24496>)

Author: Oana Balmau, Saurav Banka, Jiaxuan Chen, Anne-Marie Kermarrec, Ramya Narasimha Prabhu, Rafael Pires, Mathis Randl, Martijn de Vos

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Distributed Computing](<https://devfeed.tech/sources/arxiv-distributed-computing.md>)

Topics: [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [LLM Techniques](<https://devfeed.tech/topics/llm-techniques.md>), [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>), [Mixture of Experts (MoE)](<https://devfeed.tech/topics/mixture-of-experts-moe.md>), [GPU optimization](<https://devfeed.tech/topics/gpu-optimization.md>), [distributed-training](<https://devfeed.tech/topics/distributed-training.md>), [Multi-GPU](<https://devfeed.tech/topics/multi-gpu.md>), [GPU](<https://devfeed.tech/topics/gpu.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>)

Tags: [a100](<https://devfeed.tech/tags/a100.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [balancing](<https://devfeed.tech/tags/balancing.md>), [bubble](<https://devfeed.tech/tags/bubble.md>), [capacity](<https://devfeed.tech/tags/capacity.md>), [colocation](<https://devfeed.tech/tags/colocation.md>), [compiler](<https://devfeed.tech/tags/compiler.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-dc](<https://devfeed.tech/tags/cs-dc.md>), [cs-os](<https://devfeed.tech/tags/cs-os.md>), [cuda](<https://devfeed.tech/tags/cuda.md>), [engine](<https://devfeed.tech/tags/engine.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [expert-parallel](<https://devfeed.tech/tags/expert-parallel.md>), [expert-parallelism](<https://devfeed.tech/tags/expert-parallelism.md>), [experts](<https://devfeed.tech/tags/experts.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [gpus](<https://devfeed.tech/tags/gpus.md>), [inference](<https://devfeed.tech/tags/inference.md>), [language-model](<https://devfeed.tech/tags/language-model.md>), [large-language-model](<https://devfeed.tech/tags/large-language-model.md>)

### AI overview

BubbleTea uses per-GPU idle periods created by uneven expert-parallel token routing to run fine-tuning alongside LLM serving. It predicts each GPU's available bubble at every layer and schedules short fine-tuning chunks that reuse serving weights; compiler passes derive the chunks from the served model without porting its architecture to a separate training engine. On three MoE models served across four A100 GPUs, BubbleTea delivered 1.9-6.2x the fine-tuning throughput of LLMStation and kept p90 inference latency within 10% of serving alone in 50 of 63 workloads, versus 44 for LLMStation. On bursty production traces, it trained 3.3x faster than LLMStation and 14x faster than a stock trainer under CUDA MPS, with p90 latency within 7% of serving alone.

### Source excerpt

arXiv:2611.24496v1 Announce Type: cross Abstract: GPUs are expensive, yet latency-critical large language model (LLM) inference leaves part of them idle. Colocation reclaims this capacity by running best-effort work, such as fine-tuning, alongside serving. Many leading LLMs now use a Mixture-of-Experts (MoE) architecture, which adds a new source of idle time. Under expert parallelism, MoE experts are spread across GPUs, and because the router sends uneven numbers of tokens to each GPU, lightly loaded GPUs wait for the most loaded one. Even after applying load balancing solutions, such bubbles still take 8--13% of prefill time. We present BubbleTea, the first colocation system that leverages expert-parallel bubbles for fine-tuning. Unlike existing colocation systems, which make one scheduling decision for all GPUs per forward pass, BubbleTea predicts each GPU's bubble at every layer from the router's token assignment, and fills it with short chunks of fine-tuning work that reuse the weights already loaded for serving. We also introduce compiler passes that automatically derive these chunks from the served LLM as-is, without porting each model architecture to a separate training engine. Our evaluation on three MoE models, served on four A100 GPUs, shows that fine-tuning throughput with BubbleTea is 1.9--6.2$\times$ faster than with LLMStation, a state-of-the-art colocation system. BubbleTea keeps p90 inference latency within 10% of serving alone in 50 of 63 workloads, compared to 44 for LLMStation. On bursty production traces, BubbleTea trains 3.3$\times$ faster than LLMStation and 14$\times$ faster than a stock trainer colocated under CUDA MPS, with p90 inference latency staying within 7% of serving alone.

[Next page](<https://devfeed.tech/topics/llm-techniques.md?cursor=WyIyMDI2LTExLTE5VDA1OjAwOjAwKzAwOjAwIiwgIjA3ZDE1ODY3LTUzMDktNDg2ZS04OTBhLWZkZDM1Mjc5NDhlNCJd>)