# Training AI Models

The machine learning process of adjusting an AI model's parameters from examples.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## Antom 3-in-1 Transformer: A Payment Foundation Model

DevFeed: [Antom 3-in-1 Transformer: A Payment Foundation Model](<https://devfeed.tech/articles/antom-3-in-1-transformer-a-payment-foundation-model-145051.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.25403>)

Author: Weiyu Han, Haoyu Jiang, Jie Sheng, Youxi Pan, Xianglong Liu, Zeqi Ren, Dongquan Yi, Yinhao Liu, Ruobing Zuo, Ze Zhao, Huilin He, Nairong Zhang, Xinyi He, Xiao Tan, Jun Wang, Pan Fang, Wei Zeng, Jianan

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [AI Foundation Models](<https://devfeed.tech/topics/ai-foundation-models.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Training AI Models](<https://devfeed.tech/topics/training-ai-models.md>)

Tags: [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [experiment](<https://devfeed.tech/tags/experiment.md>), [foundation-model](<https://devfeed.tech/tags/foundation-model.md>), [graph](<https://devfeed.tech/tags/graph.md>), [model](<https://devfeed.tech/tags/model.md>), [online](<https://devfeed.tech/tags/online.md>), [payment](<https://devfeed.tech/tags/payment.md>), [pretraining](<https://devfeed.tech/tags/pretraining.md>), [risk](<https://devfeed.tech/tags/risk.md>), [sequence](<https://devfeed.tech/tags/sequence.md>), [transformer](<https://devfeed.tech/tags/transformer.md>)

### AI overview

Antom 3-in-1 Transformer combines sequence, tabular, and graph representations to detect payment risk. The approximately 7B-parameter model, with about 1.7B activated per task route, is pretrained mainly on Ant International internal payment data to learn reusable behavioral and relational representations. In reported experiments, pretraining raised a target business metric 34.5% over the same architecture trained from scratch; with 25% of training labels, it exceeded the full-label scratch baseline on Average Precision and Recall@0.5% across merchant domains. A four-month online experiment on Ant International's billion-scale platform reported a 37.88% lower overall loss rate than XGB-Online.

### Source excerpt

arXiv:2611.25403v1 Announce Type: new Abstract: We report the development of Antom 3-in-1 Transformer, a payment foundation model that integrates sequence, tabular, and graph representations for payment risk detection. The model contains approximately 7B parameters, with approximately 1.7B activated per task route. Antom 3-in-1 Transformer is pretrained to learn reusable behavioral and relational representations, primarily on Ant International's internal payment data. In a cross-business transfer experiment, pretraining improves the target business metric by 34.5% over the same architecture trained from scratch. With only 25% of the training labels, the pretrained model outperforms the full-label scratch baseline in Average Precision (AP) and Recall@0.5% on different merchant domains. In a four-month online experiment on Ant International's billion-scale payment platform, the model reduced the overall loss rate by 37.88% relative to XGB-Online.

## GLiDE: A Fast, Calibrated Decision Model Distilled from Reasoning Teachers

DevFeed: [GLiDE: A Fast, Calibrated Decision Model Distilled from Reasoning Teachers](<https://devfeed.tech/articles/glide-a-fast-calibrated-decision-model-distilled-from-reasoning-teachers-145678.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.26509>)

Author: Sahibzada Allahyar, Julia White, Dhruv Atreja, George Hurn-Maloney, Urchade Zaratiana

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [Training AI Models](<https://devfeed.tech/topics/training-ai-models.md>), [Post-training optimization](<https://devfeed.tech/topics/post-training-optimization.md>), [model architecture](<https://devfeed.tech/topics/model-architecture.md>), [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>), [qwen](<https://devfeed.tech/topics/qwen.md>)

Tags: [attention](<https://devfeed.tech/tags/attention.md>), [calibrated-decision-model](<https://devfeed.tech/tags/calibrated-decision-model.md>), [confidence](<https://devfeed.tech/tags/confidence.md>), [context](<https://devfeed.tech/tags/context.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [decision](<https://devfeed.tech/tags/decision.md>), [decision-model](<https://devfeed.tech/tags/decision-model.md>), [decision-models](<https://devfeed.tech/tags/decision-models.md>), [di](<https://devfeed.tech/tags/di.md>), [distillation](<https://devfeed.tech/tags/distillation.md>), [fast](<https://devfeed.tech/tags/fast.md>), [inference](<https://devfeed.tech/tags/inference.md>), [language](<https://devfeed.tech/tags/language.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [latency](<https://devfeed.tech/tags/latency.md>), [model](<https://devfeed.tech/tags/model.md>), [performance](<https://devfeed.tech/tags/performance.md>), [post-training](<https://devfeed.tech/tags/post-training.md>), [qwen](<https://devfeed.tech/tags/qwen.md>), [reasoning](<https://devfeed.tech/tags/reasoning.md>), [reasoning-model](<https://devfeed.tech/tags/reasoning-model.md>)

### AI overview

GLiDE transfers reasoning capability into single-pass decision models that score candidate answers without generating text or intermediate reasoning. The 0.4B-27B family uses Qwen hybrid language models with bidirectional interactions among context, question, and candidates, plus soft-target distillation from multiple teachers--including a reasoning model--and progressive post-training on varied decision tasks. Parameter-free recurrent depth adds inference-time computation without token generation. The abstract reports that each of five sizes outperforms evaluated alternatives of equal or smaller size on the Hugging Face benchmark. The 27B model scores 71.52 on a hidden set with 2.0% expected calibration error and 92.9 ms median latency on one GPU; checkpoint averaging and recurrent depth raise the score to 73.37, with 2.1% calibration error.

### Source excerpt

arXiv:2611.26509v1 Announce Type: new Abstract: Decision models enable fast, confidence-aware decisions by scoring candidate answers in a single forward pass, without generating text or intermediate reasoning. Yet reasoning can substantially improve decision quality, raising the question of whether its benefits can be retained without the cost of generating it. We investigate this question with \textbf{GLiDE}, a family of decision models spanning 0.4B to 27B parameters. Building on Qwen hybrid language models, GLiDE enables bidirectional interactions between the context, question, and candidate answers by removing causal masking from full-attention layers. To transfer reasoning capabilities into single-pass inference, we combine soft-target distillation from multiple teachers, including a reasoning model, with progressive post-training on diverse decision tasks. We further explore parameter-free recurrent depth to increase inference-time computation without generating tokens. Across five model sizes, GLiDE lies on the performance--parameter Pareto frontier of the Hugging Face \DI{}, with each model outperforming evaluated alternatives of equal or smaller size. \dmtwo{} (27B) achieves a hidden-set score of 71.52, with an expected calibration error of 2.0\% and a median latency of 92.9\,ms on a single GPU. Further checkpoint averaging and parameter-free recurrent depth yield GLiDE~27B, which reaches 73.37 with an expected calibration error of 2.1\%.

## C-LiTE: A Lightweight Foundation Model for Cellular feature learning in Histopathology

DevFeed: [C-LiTE: A Lightweight Foundation Model for Cellular feature learning in Histopathology](<https://devfeed.tech/articles/c-lite-a-lightweight-foundation-model-for-cellular-feature-learning-in-histopathology-145653.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.26316>)

Author: Esha Sadia Nasir, Jiaqi Lv, Gozde N. Gunesli, Shan E Ahmed Raza

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [AI Foundation Models](<https://devfeed.tech/topics/ai-foundation-models.md>), [Representation learning](<https://devfeed.tech/topics/representation-learning.md>), [Training AI Models](<https://devfeed.tech/topics/training-ai-models.md>), [Dataset development](<https://devfeed.tech/topics/dataset-development.md>), [image recognition](<https://devfeed.tech/topics/image-recognition.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>)

Tags: [cellular](<https://devfeed.tech/tags/cellular.md>), [classification](<https://devfeed.tech/tags/classification.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [embedding](<https://devfeed.tech/tags/embedding.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [f1](<https://devfeed.tech/tags/f1.md>), [features](<https://devfeed.tech/tags/features.md>), [fewer](<https://devfeed.tech/tags/fewer.md>), [figure](<https://devfeed.tech/tags/figure.md>), [foundation-model](<https://devfeed.tech/tags/foundation-model.md>), [lightweight](<https://devfeed.tech/tags/lightweight.md>), [native](<https://devfeed.tech/tags/native.md>), [public](<https://devfeed.tech/tags/public.md>), [representation-learning](<https://devfeed.tech/tags/representation-learning.md>), [token](<https://devfeed.tech/tags/token.md>), [training-dataset](<https://devfeed.tech/tags/training-dataset.md>), [transformation](<https://devfeed.tech/tags/transformation.md>), [type](<https://devfeed.tech/tags/type.md>), [vit](<https://devfeed.tech/tags/vit.md>)

### AI overview

C-LiTE is a 22-million-parameter model for learning cellular features from H&E histopathology images. It processes 64x64-pixel cell-centred crops at native resolution, using 4x4-pixel patches to form a 16x16 token grid without upsampling; its patch embedding is initialized by adapting weights from a pre-trained ViT-5-S with a pseudoinverse-based transformation. The model is pre-trained on 60 million TCGA cell crops balanced across 19 cancer types and 16 tissue types. Across seven public datasets for mitotic figure recognition, nucleus and cell-type classification, and immune-cell phenotyping, frozen-feature evaluation yields mean macro-F1 0.670, close to CytoFormer's 0.682 with about 30 times fewer parameters. These findings are limited to the reported evaluation setting.

### Source excerpt

arXiv:2611.26316v1 Announce Type: cross Abstract: Characterising individual cells is fundamental to computational pathology, as cellular morphology informs tasks such as tumour grading, mitotic assessment and immune profiling. However, most pathology foundation models are trained on tissue patches and capture cellular features only implicitly, while existing cell-level models are either restricted to specific cell populations or rely on computationally expensive architectures with hundreds of millions of parameters. Applying ViTs directly to small cellular crops also presents an architectural challenge: upsampling increases computational cost without adding morphological information and may introduce interpolation artefacts. To address these limitations, we introduce C-LiTE, a lightweight foundation model for cellular representation learning in H&E-stained histopathology, with only 22 million parameters. C-LiTE employs a native-resolution tokenisation strategy tailored to cellular inputs, processing 64 x 64 px cell-centred crops using a 4 x 4 patch embedding to produce a 16 x 16 token grid without image upsampling. To transfer visual knowledge from large-scale pre-training, we adapt the patch embedding weights of a pre-trained ViT-5-S using a pseudoinverse-based transformation, enabling weight initialisation at the finer patch resolution. We further assemble a pre-training dataset of 60 million cell-centred crops from TCGA, balanced across 19 cancer types and 16 tissue types. To our knowledge, this represents the largest public dataset for self-supervised pre-training on H&E images at the cellular level. We evaluate C-LiTE on seven public datasets spanning mitotic figure recognition, nucleus and cell-type classification, and immune cell phenotyping. Under frozen-feature evaluation, C-LiTE achieves a mean macro-F1 of 0.670, approaching CytoFormer (0.682) with approximately 30x fewer parameters.

## Xaurora: Generative Weather Forecasting with Denoising Stochastic Interpolants from a Foundation Model Prior

DevFeed: [Xaurora: Generative Weather Forecasting with Denoising Stochastic Interpolants from a Foundation Model Prior](<https://devfeed.tech/articles/xaurora-generative-weather-forecasting-with-denoising-stochastic-interpolants-from-a-foundation-model-prior-144958.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.06509>)

Author: Eliot Walt, Miltiadis Kofinas, Nikolaj M\\"ucke, Efstratios Gavves, Dim Coumou

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Climate / Weather / Ocean Modeling](<https://devfeed.tech/topics/climate-weather-ocean-modeling.md>), [AI Foundation Models](<https://devfeed.tech/topics/ai-foundation-models.md>), [Training AI Models](<https://devfeed.tech/topics/training-ai-models.md>)

Tags: [buffer](<https://devfeed.tech/tags/buffer.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [extreme-weather](<https://devfeed.tech/tags/extreme-weather.md>), [forecasting](<https://devfeed.tech/tags/forecasting.md>), [foundation-model](<https://devfeed.tech/tags/foundation-model.md>), [generative](<https://devfeed.tech/tags/generative.md>), [global](<https://devfeed.tech/tags/global.md>), [has](<https://devfeed.tech/tags/has.md>), [large](<https://devfeed.tech/tags/large.md>), [novel](<https://devfeed.tech/tags/novel.md>), [parameter](<https://devfeed.tech/tags/parameter.md>), [physics-ao-ph](<https://devfeed.tech/tags/physics-ao-ph.md>), [probabilistic](<https://devfeed.tech/tags/probabilistic.md>), [range](<https://devfeed.tech/tags/range.md>), [sample](<https://devfeed.tech/tags/sample.md>), [sde](<https://devfeed.tech/tags/sde.md>), [skill](<https://devfeed.tech/tags/skill.md>), [small](<https://devfeed.tech/tags/small.md>), [train](<https://devfeed.tech/tags/train.md>), [uncertainty-quantification](<https://devfeed.tech/tags/uncertainty-quantification.md>), [weather](<https://devfeed.tech/tags/weather.md>), [weather-forecasting](<https://devfeed.tech/tags/weather-forecasting.md>)

### AI overview

Xaurora extends the pretrained Aurora weather foundation model into a stochastic ensemble forecaster, addressing deterministic forecasts' limits for uncertainty estimates and the cost of training a model from scratch. It is finetuned from Aurora's small version using Denoising Stochastic Interpolants and a replay buffer for Stochastic Differential Equation rollouts, which the abstract says enables probabilistic trajectory training. The authors report that Xaurora approaches state of the art on global ensemble metrics, is competitive with Aurora's large version, and produces skilful 15-day forecasts in seven minutes. The abstract characterizes the method as parameter- and sample-efficient; it provides no further evaluation details here.

### Source excerpt

arXiv:2610.06509v2 Announce Type: replace Abstract: Deep learning has revolutionised weather forecasting in recent years, especially through atmospheric foundation models, which offer competitive skill for a fraction of the computational costs of classic physics-based models. However, most existing foundation models are deterministic, limiting the generation of large ensembles for accurate uncertainty quantification, extreme weather risk assessment, and long-range weather forecasting. Furthermore, these models incur a large, often prohibitive, computational overhead to train from scratch. To address these shortcomings, we turn a pretrained deterministic prior model, namely the Aurora foundation model, into a generative ensemble-prediction model. To that end, we introduce a novel generative method, Denoising Stochastic Interpolants, combined with a replay buffer for Stochastic Differential Equation (SDE) rollout, enabling probabilistic training of SDE trajectories. Our stochastic foundation model, Xaurora, is finetuned from the small Aurora version, yet it approaches the state-of-the-art on global ensemble metrics and is competitive with the large version of Aurora. Our method is parameter and sample efficient, and generates skilful 15-day forecasts in 7 minutes. Our results demonstrate that deterministic foundation models can be efficiently extended into even stronger stochastic models.

## Model Compression Beyond Quantization

DevFeed: [Model Compression Beyond Quantization](<https://devfeed.tech/articles/model-compression-beyond-quantization-143867.md>)

Original publisher: [Read original article](<https://machinelearningmastery.com/model-compression-beyond-quantization/>)

Author: Adrian Tam

Published: 2026-11-19T14:06:50Z

Content type: article

Language: en

Sources: [MachineLearningMastery.com](<https://devfeed.tech/sources/machinelearningmastery-com.md>)

Topics: [Post-training optimization](<https://devfeed.tech/topics/post-training-optimization.md>), [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>), [Neural Network](<https://devfeed.tech/topics/neural-network.md>), [model architecture](<https://devfeed.tech/topics/model-architecture.md>), [Training AI Models](<https://devfeed.tech/topics/training-ai-models.md>)

Tags: [compression](<https://devfeed.tech/tags/compression.md>), [distillation](<https://devfeed.tech/tags/distillation.md>), [inference-from-transformer-models](<https://devfeed.tech/tags/inference-from-transformer-models.md>), [knowledge-distillation](<https://devfeed.tech/tags/knowledge-distillation.md>), [model-compression](<https://devfeed.tech/tags/model-compression.md>), [pruning](<https://devfeed.tech/tags/pruning.md>), [quantization](<https://devfeed.tech/tags/quantization.md>), [sparsity](<https://devfeed.tech/tags/sparsity.md>), [structured](<https://devfeed.tech/tags/structured.md>)

### AI overview

Compression methods can reduce model size by removing weights or structures, or by training a smaller model to imitate a larger one. Unstructured magnitude pruning zeros low-magnitude weights, but speed or storage gains depend on sparse representation, matrix shape, sparsity, dtype, hardware, and kernel support. Structured pruning removes coordinated units such as MLP channels; matching dimensions must be edited together, and hardware-friendly sizes matter. Knowledge distillation trains a student from teacher signals such as output distributions or intermediate representations. It requires teacher targets, compatible logits and token positions, and representative training data; generating targets has a cost. The text stresses evaluating held-out quality, memory, latency, and throughput, since reduced parameter counts do not guarantee preserved behavior or faster serving.

### Source excerpt

Overview This post is divided into three parts; they are: - Pruning and Sparsity - Structured Pruning - Knowledge Distillation Pruning and Sparsity Pruning sets selected model parameters to zero or removes larger structures.

## AllotNet: an Interpretable Mixture-of-Experts Choice Model for Moral Judgement in Humanitarian Aid Allocation, Validated against a Planted Synthetic Population

DevFeed: [AllotNet: an Interpretable Mixture-of-Experts Choice Model for Moral Judgement in Humanitarian Aid Allocation, Validated against a Planted Synthetic Population](<https://devfeed.tech/articles/allotnet-an-interpretable-mixture-of-experts-choice-model-for-moral-judgement-in-humanitarian-aid-allocation-validated-against-a-planted-synthetic-population-142660.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.23474>)

Author: Mark Nowak (Conflux Laboratory)

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Mixture of Experts (MoE)](<https://devfeed.tech/topics/mixture-of-experts-moe.md>), [Training AI Models](<https://devfeed.tech/topics/training-ai-models.md>), [ai alignment](<https://devfeed.tech/topics/ai-alignment.md>)

Tags: [bayesian](<https://devfeed.tech/tags/bayesian.md>), [budget](<https://devfeed.tech/tags/budget.md>), [choice](<https://devfeed.tech/tags/choice.md>), [code](<https://devfeed.tech/tags/code.md>), [conditional](<https://devfeed.tech/tags/conditional.md>), [context](<https://devfeed.tech/tags/context.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cy](<https://devfeed.tech/tags/cs-cy.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [encoder](<https://devfeed.tech/tags/encoder.md>), [failure](<https://devfeed.tech/tags/failure.md>), [gate](<https://devfeed.tech/tags/gate.md>), [generation](<https://devfeed.tech/tags/generation.md>), [hidden](<https://devfeed.tech/tags/hidden.md>), [java](<https://devfeed.tech/tags/java.md>), [mixture-of-experts](<https://devfeed.tech/tags/mixture-of-experts.md>), [nvfp4](<https://devfeed.tech/tags/nvfp4.md>), [planner](<https://devfeed.tech/tags/planner.md>), [python](<https://devfeed.tech/tags/python.md>), [queue](<https://devfeed.tech/tags/queue.md>), [rank](<https://devfeed.tech/tags/rank.md>), [ranking](<https://devfeed.tech/tags/ranking.md>), [synthetic](<https://devfeed.tech/tags/synthetic.md>)

### AI overview

AllotNet combines eight interpretable taste experts over seven ethical criteria with participant-context weighting to model allocation judgements elicited by the pre-registered, six-language Allot instrument. Its budget-constrained planner creates a review queue and never rejects anyone. On a planted synthetic population of 5.6 million sessions, the authors report that all 16 pre-specified checks passed, including criterion-ranking correlation of 1.000 and planner performance at 99.4% of an oracle. They also flag that two checks are weaker than their titles suggest and that a generation failure cost 260,712 seeds. The 23,550-parameter model has a 125 KB bundle and C11, Java, Rust, and Python runtimes. No real data has been used for training; a hierarchical Bayesian mixed logit is the registered comparator at the study's real scale.

### Source excerpt

arXiv:2611.23474v1 Announce Type: cross Abstract: Official development assistance fell by 23.1 per cent in 2025, and the decision of who is helped first when there is not enough for everyone is moving into software. A leading published machine-learning targeting system excluded 53 per cent of eligible households as deployed. If the rule of distribution ends up in code, it should carry the judgement of many cultures, and the machine should execute while a person judges. AllotNet is an interpretable mixture-of-experts conditional logit for the allocation judgements elicited by Allot, a pre-registered six-language instrument: eight linear taste experts over seven ethical criteria, weighted by a gate over participant context, with a damped residual, an encoder for household intake, an outcome head and a budget-constrained planner that builds a review queue and never rejects anyone. A model of human judgement has no answer key on real data, so we wrote one first: a planted synthetic population of 5.6 million sessions and 104 million responses (98 million usable choices), generated by the production code path from a ground truth hidden from the loss. All sixteen pre-specified checks passed: criterion ranking $\rho = 1.000$, hidden segments recovered (ARI 0.671), outcome head at 99 per cent of the noise ceiling, planner at 99.4 per cent of oracle. The caveats are reported with them: two checks are weaker than their titles, and a live-generation failure cost 260,712 seeds. The model has 23,550 parameters, is trained quantisation-aware for NVFP4 with its interpretable core in fp32 (welfare-rank parity $\rho = 0.9937$ at W4A16 on the shipped bundle; one parity threshold was relaxed post hoc), and ships as a 125 KB bundle with C11, Java, Rust and Python runtimes. Nothing has been trained on real data yet; at the study's real scale a hierarchical Bayesian mixed logit is the registered comparator.

## Learning through Discernment: when a small language model learns from user feedback on the job, and when it decays

DevFeed: [Learning through Discernment: when a small language model learns from user feedback on the job, and when it decays](<https://devfeed.tech/articles/learning-through-discernment-when-a-small-language-model-learns-from-user-feedback-on-the-job-and-when-it-decays-142695.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.23716>)

Author: Sirisilp Kongsilp

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Training AI Models](<https://devfeed.tech/topics/training-ai-models.md>), [Language models](<https://devfeed.tech/topics/language-models.md>), [lora](<https://devfeed.tech/topics/lora.md>), [rlhf](<https://devfeed.tech/topics/rlhf.md>), [On-device AI](<https://devfeed.tech/topics/on-device-ai.md>)

Tags: [adapter](<https://devfeed.tech/tags/adapter.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [feedback](<https://devfeed.tech/tags/feedback.md>), [job](<https://devfeed.tech/tags/job.md>), [language-model](<https://devfeed.tech/tags/language-model.md>), [learning](<https://devfeed.tech/tags/learning.md>), [small-language-model](<https://devfeed.tech/tags/small-language-model.md>), [user](<https://devfeed.tech/tags/user.md>)

### AI overview

In experiments with Qwen3 0.6B and 1.7B plus LoRA adapters, nightly learning from user feedback depended on whether the judge filtered out failures and whether users could see outcomes. A feedback sense that treated polite give-ups as success collapsed in 3 of 5 lives. A version that first checked for a clear user verdict stayed near oracle performance in 4 of 5 lives for two weeks, then faded to self-judging levels by day 60 as clean lessons became scarce. On τ²-bench, it interpreted reactions correctly, but 80% of lessons were failed attempts users had thanked; across 97 lives, the share of failed lessons strongly predicted outcomes (Spearman ρ = −0.81). The abstract concludes that user-guided learning held up only when failures were excluded and users could observe outcomes, and warns that skewed lessons may narrow the model.

### Source excerpt

arXiv:2611.23716v1 Announce Type: cross Abstract: Small language models on devices could keep improving by learning, overnight, from their own daily work. Without a true outcome, however, something must decide which attempts are worth learning from, and both the model's own judgment and the user's reaction can be wrong. We study a small model (Qwen3 0.6B and 1.7B) with a LoRA adapter, fine-tuned each night on the attempts a judge accepts, over 127 lives of up to a year on WebShop and $\tau^2$-bench. We compare the true outcome, the user's stated verdict and the model's own judgment with a separate, frozen "feedback sense" that reads the user's reaction, built in three versions. Judging itself, the model improved and then decayed, over a year and whatever its adapter. A sense that took polite give-ups for success collapsed 3 of 5 lives; one built for discernment, which first asks whether the user gave a clear verdict, kept 4 of 5 near the oracle for two weeks, then faded on clean but scarce lessons, to the level of self-judging by day 60. When the user cannot see the outcome ($\tau^2$-bench), the sense read reactions correctly, yet 80% of its lessons were failures the customer had thanked. Across 97 lives, the share of failed attempts among the lessons predicted the outcome (Spearman $\rho$ = -0.81). In our experiments, learning from users held up only where the judge kept failures out and the user could see the outcome; lopsided lessons may also narrow the model.

## DuoCL: Dual-Path Online Continual Learning

DevFeed: [DuoCL: Dual-Path Online Continual Learning](<https://devfeed.tech/articles/duocl-dual-path-online-continual-learning-143410.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.23598>)

Author: Jebacyril Arockiaraj, Dhruv Parikh, Rajgopal Kannan, Viktor Prasanna

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Training AI Models](<https://devfeed.tech/topics/training-ai-models.md>), [Representation learning](<https://devfeed.tech/topics/representation-learning.md>)

Tags: [alignment](<https://devfeed.tech/tags/alignment.md>), [budget](<https://devfeed.tech/tags/budget.md>), [continual-learning](<https://devfeed.tech/tags/continual-learning.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [data](<https://devfeed.tech/tags/data.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [distribution](<https://devfeed.tech/tags/distribution.md>), [distributions](<https://devfeed.tech/tags/distributions.md>), [domain](<https://devfeed.tech/tags/domain.md>), [drift](<https://devfeed.tech/tags/drift.md>), [fast](<https://devfeed.tech/tags/fast.md>), [feature](<https://devfeed.tech/tags/feature.md>), [framework](<https://devfeed.tech/tags/framework.md>), [gradient](<https://devfeed.tech/tags/gradient.md>), [incremental](<https://devfeed.tech/tags/incremental.md>), [model](<https://devfeed.tech/tags/model.md>), [model-training](<https://devfeed.tech/tags/model-training.md>), [online](<https://devfeed.tech/tags/online.md>), [overhead](<https://devfeed.tech/tags/overhead.md>), [points](<https://devfeed.tech/tags/points.md>), [reduce](<https://devfeed.tech/tags/reduce.md>)

### AI overview

DuoCL combines a fast analytic path that updates feature statistics on every incoming minibatch with a slower path that periodically adapts the backbone. The slow path adjusts how many samples it processes to fit the available compute budget, while feature alignment compensates for drift in classifier statistics. Across four datasets and three compute budgets, the authors report an average 9.67 percentage point online accuracy gain over the strongest gradient-based baseline in each setting. Under distribution shifts, DuoCL also outperforms analytic learning, with gains up to 4.00 points on CIFAR-C, and reports higher accuracy than prior methods in disjoint class-incremental, domain-incremental, and Si-Blurry settings.

### Source excerpt

arXiv:2611.23598v1 Announce Type: new Abstract: Online continual learning (OCL) requires learning from a data stream whose arrival rate is independent of model training speed. When training cannot keep pace, computationally expensive gradient-based methods update less frequently, limiting adaptation. Analytic methods reduce this overhead, but their frozen backbones cannot adapt representations to changing distributions. We propose DuoCL, a dual-path OCL framework whose fast analytic path accumulates feature statistics from every incoming minibatch while its slow path periodically adapts the backbone. The slow path adjusts its sample count to the available compute budget without interrupting fast-path learning. Feature alignment compensates for drift in the accumulated classifier statistics. Across four datasets and three computational budgets, DuoCL improves online accuracy by an average of 9.67 percentage points over the strongest gradient-based baseline in each setting. DuoCL consistently outperforms analytic learning under distribution shifts, with gains of up to 4.00 percentage points on CIFAR-C, and achieves higher accuracy than prior methods across disjoint class-incremental, domain-incremental, and Si-Blurry settings.

## Survival of the Fittest Perturbations: Uncertainty-Aware Evolution of Adversarial Distributions for Robust Generalization

DevFeed: [Survival of the Fittest Perturbations: Uncertainty-Aware Evolution of Adversarial Distributions for Robust Generalization](<https://devfeed.tech/articles/survival-of-the-fittest-perturbations-uncertainty-aware-evolution-of-adversarial-distributions-for-robust-generalization-143217.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2411.02871>)

Author: Junhao Dong, Zicheng Yu, Xinghua Qu, Z. Jane Wang, Yew-Soon Ong

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Adversarial attacks](<https://devfeed.tech/topics/adversarial-attacks.md>), [Training AI Models](<https://devfeed.tech/topics/training-ai-models.md>)

Tags: [adversarial](<https://devfeed.tech/tags/adversarial.md>), [alignment](<https://devfeed.tech/tags/alignment.md>), [analysis](<https://devfeed.tech/tags/analysis.md>), [anchor](<https://devfeed.tech/tags/anchor.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [certificate](<https://devfeed.tech/tags/certificate.md>), [component](<https://devfeed.tech/tags/component.md>), [conditional](<https://devfeed.tech/tags/conditional.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [deep-learning](<https://devfeed.tech/tags/deep-learning.md>), [deployment](<https://devfeed.tech/tags/deployment.md>), [design](<https://devfeed.tech/tags/design.md>), [distribution](<https://devfeed.tech/tags/distribution.md>), [distributions](<https://devfeed.tech/tags/distributions.md>), [effectiveness](<https://devfeed.tech/tags/effectiveness.md>), [elite](<https://devfeed.tech/tags/elite.md>), [evaluations](<https://devfeed.tech/tags/evaluations.md>), [evolution](<https://devfeed.tech/tags/evolution.md>), [examples](<https://devfeed.tech/tags/examples.md>), [robust](<https://devfeed.tech/tags/robust.md>), [uncertainty](<https://devfeed.tech/tags/uncertainty.md>)

### AI overview

EvoUAD-AT addresses a limitation of pointwise adversarial training, which assigns one adversary to each clean sample and misses variation in adversarial feature statistics. It maintains an instance-specific search distribution around an adversarial anchor, evaluates candidates under the current model, selects those most threatening to a benign reference, and adapts the distribution while retaining strategy parameters across epochs. It also refines the alignment reference to avoid misclassified clean samples, matches benign and adversarial distributions, and uses introspective gradient matching without external supervision. The authors report that evaluations across multiple benchmarks show improved clean accuracy and robustness over existing state-of-the-art methods; the supplied abstract gives no benchmark names or quantitative results.

### Source excerpt

arXiv:2411.02871v2 Announce Type: replace Abstract: Despite deep learning's broad successes across diverse scenarios, its vulnerability to adversarial examples impedes deployment, making adversarial training a leading approach to improve robustness. However, pointwise adversarial training maps each clean sample to a single adversary, overlooking surrounding adversarial variability and the associated shifts in feature statistics. To address these issues, we propose Evolutionary Uncertainty-Aware Distributional Adversarial Training (EvoUAD-AT). Specifically, we design an evolutionary adversarial uncertainty modeling module that maintains an instance-wise search distribution over adversarial feature statistics around an adversarially generated anchor. Candidates are evaluated under the current model, selected by their threat to a benign reference, and used to adapt the distribution; strategy parameters are retained across epochs. To avoid anchoring on misclassified clean samples, we refine the alignment reference and match benign and adversarial distributions. Introspective gradient matching further regularizes input gradients without external supervision. Our analysis characterizes the elite fitness surrogate and gives a conditional certificate for latent adversarial features. Extensive evaluations across multiple benchmarks demonstrate that our approach consistently outperforms existing state-of-the-art methods in both clean accuracy and robustness, with thorough analyses confirming the effectiveness of each proposed component.

## Inherited Adapter Experts for Replay-Free Domain-Incremental Learning

DevFeed: [Inherited Adapter Experts for Replay-Free Domain-Incremental Learning](<https://devfeed.tech/articles/inherited-adapter-experts-for-replay-free-domain-incremental-learning-142758.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.24145>)

Author: Gyutae Oh, Hyunmin Kong, Jitae Shin

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Mixture of Experts (MoE)](<https://devfeed.tech/topics/mixture-of-experts-moe.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Training AI Models](<https://devfeed.tech/topics/training-ai-models.md>), [image recognition](<https://devfeed.tech/topics/image-recognition.md>)

Tags: [adapter](<https://devfeed.tech/tags/adapter.md>), [adapters](<https://devfeed.tech/tags/adapters.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [computer-vision](<https://devfeed.tech/tags/computer-vision.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [deepfake](<https://devfeed.tech/tags/deepfake.md>), [domain](<https://devfeed.tech/tags/domain.md>), [domains](<https://devfeed.tech/tags/domains.md>), [experts](<https://devfeed.tech/tags/experts.md>), [free](<https://devfeed.tech/tags/free.md>), [incremental](<https://devfeed.tech/tags/incremental.md>), [learning](<https://devfeed.tech/tags/learning.md>), [points](<https://devfeed.tech/tags/points.md>), [router](<https://devfeed.tech/tags/router.md>), [vit](<https://devfeed.tech/tags/vit.md>)

### AI overview

Lamarck addresses replay-free domain-incremental learning by strengthening per-domain experts rather than training a router: the abstract reports that a training-free Mahalanobis router costs the analyzed learners under 2.7 points versus always choosing the own-domain expert. Lamarck adds adapters after each deep ViT block, improving final accuracy by 4.4 points on CDDB-Hard, and initializes each new expert from its predecessor at no extra cost. This is intended to reduce the cost when an image is routed to a later expert, since domains share classes. Against twelve frozen-backbone baselines under one protocol, Lamarck ranks first on CDDB-Hard by 5.2 points, second on DomainNet and OfficeHome, and highest in mean accuracy.

### Source excerpt

arXiv:2611.24145v1 Announce Type: cross Abstract: Replay-free domain-incremental learning trains a model on the same classes in successive domains without keeping earlier images. Per-domain learners freeze a pre-trained ViT and train an additional expert for each new domain: the small set of parameters and the classification head learned on that domain. Test images carry no domain label, so a router selects one expert for each image. Recent work trains this router. However, a training-free Mahalanobis router costs the analyzed learners under 2.7 points of final accuracy relative to always selecting each image's own-domain expert. The experts, not this router, limit accuracy. Our learner, Lamarck, therefore uses this router and strengthens the experts. Its adapters, small trainable layers that correct every image token after each deep ViT block, add 4.4 points of final accuracy on the deepfake benchmark CDDB-Hard. In addition, each expert is initialized from its predecessor at no extra cost: the domains share their classes, so an image misrouted to a later expert loses less. Against twelve frozen-backbone baselines under one protocol, Lamarck is first on CDDB-Hard by 5.2 points, second on DomainNet and OfficeHome, and highest in mean accuracy.

## How to delete data from a trained AI model (when DELETE doesn't work)

DevFeed: [How to delete data from a trained AI model (when DELETE doesn't work)](<https://devfeed.tech/articles/how-to-delete-data-from-a-trained-ai-model-when-delete-doesn-t-work-141074.md>)

Original publisher: [Read original article](<https://www.red-gate.com/simple-talk/ai/delete-data-from-trained-ai-model/>)

Author: Dejan Lukić

Published: 2026-11-18T12:00:00Z

Content type: article

Language: en

Sources: [Simple Talk](<https://devfeed.tech/sources/simple-talk.md>)

Topics: [AI Models](<https://devfeed.tech/topics/ai-models.md>), [Training AI Models](<https://devfeed.tech/topics/training-ai-models.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [ai-model](<https://devfeed.tech/tags/ai-model.md>), [ai-training](<https://devfeed.tech/tags/ai-training.md>), [anthropic](<https://devfeed.tech/tags/anthropic.md>), [api](<https://devfeed.tech/tags/api.md>), [ccpa](<https://devfeed.tech/tags/ccpa.md>), [chatgpt](<https://devfeed.tech/tags/chatgpt.md>), [checkpoint](<https://devfeed.tech/tags/checkpoint.md>), [children](<https://devfeed.tech/tags/children.md>), [churn](<https://devfeed.tech/tags/churn.md>), [consumer](<https://devfeed.tech/tags/consumer.md>), [customer-data](<https://devfeed.tech/tags/customer-data.md>), [data](<https://devfeed.tech/tags/data.md>), [data-catalog](<https://devfeed.tech/tags/data-catalog.md>), [data-masking](<https://devfeed.tech/tags/data-masking.md>), [data-processing](<https://devfeed.tech/tags/data-processing.md>), [data-protection](<https://devfeed.tech/tags/data-protection.md>), [database](<https://devfeed.tech/tags/database.md>), [database-administration](<https://devfeed.tech/tags/database-administration.md>), [databases](<https://devfeed.tech/tags/databases.md>), [delete](<https://devfeed.tech/tags/delete.md>), [featured](<https://devfeed.tech/tags/featured.md>), [gdpr](<https://devfeed.tech/tags/gdpr.md>), [how-to](<https://devfeed.tech/tags/how-to.md>)

### AI overview

A trained model has no database row to delete: training data influences distributed weights, and personal information may still be extracted from them. The article says retraining without the record is the only guaranteed removal method. SISA retrains the affected shard and checkpoint rollback can remove data from a known fine-tuning batch; approximate unlearning and output filters cannot prove removal, and filters leave data in the weights. Under GDPR, a model is anonymous only if extraction risk is negligible; the article presents extractable personal information as a CCPA concern too. The FTC has ordered destruction of models built on improperly collected data. For erasure requests, check training manifests for affected models and derivatives, then choose a removal action; an output filter may be interim protection. To reduce future exposure, classify and minimize data, pseudonymize and mask identifiers, scrub free text, deduplicate and test for memorization, and consider differential privacy, which can cost model quality. Provider deletion of stored logs or chats does not remove prior learning; training use depends on account tier.

### Source excerpt

You can't DELETE a record from a trained AI model. Compare retraining, SISA, unlearning and filters, and see how to meet GDPR and CCPA erasure requests.... The post How to delete data from a trained AI model (when DELETE doesn't work) appeared first on Simple Talk.

## In-Warehouse Machine Learning and the Modern Data Stack | Blog | Fivetran

DevFeed: [In-Warehouse Machine Learning and the Modern Data Stack | Blog | Fivetran](<https://devfeed.tech/articles/in-warehouse-machine-learning-and-the-modern-data-stack-blog-fivetran-141332.md>)

Original publisher: [Read original article](<https://wf.fivetran.com/blog/in-warehouse-machine-learning-and-the-modern-data-stack>)

Author: Fivetran

Published: 2026-11-18T06:13:15Z

Content type: article

Language: en

Sources: [Fivetran Blog](<https://devfeed.tech/sources/fivetran-blog.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [AI Development](<https://devfeed.tech/topics/ai-development.md>), [Hyperparameter optimization](<https://devfeed.tech/topics/hyperparameter-optimization.md>), [Training AI Models](<https://devfeed.tech/topics/training-ai-models.md>), [Inference](<https://devfeed.tech/topics/inference.md>), [Amazon SageMaker](<https://devfeed.tech/topics/amazon-sagemaker.md>), [Tensorflow](<https://devfeed.tech/topics/tensorflow.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [airflow](<https://devfeed.tech/tags/airflow.md>), [analytics](<https://devfeed.tech/tags/analytics.md>), [automated-machine-learning](<https://devfeed.tech/tags/automated-machine-learning.md>), [aws](<https://devfeed.tech/tags/aws.md>), [bigquery](<https://devfeed.tech/tags/bigquery.md>), [data-engineering](<https://devfeed.tech/tags/data-engineering.md>), [data-ingestion](<https://devfeed.tech/tags/data-ingestion.md>), [data-lake](<https://devfeed.tech/tags/data-lake.md>), [data-lakehouse](<https://devfeed.tech/tags/data-lakehouse.md>), [data-lakehouse-architecture](<https://devfeed.tech/tags/data-lakehouse-architecture.md>), [data-pipelines](<https://devfeed.tech/tags/data-pipelines.md>), [data-science](<https://devfeed.tech/tags/data-science.md>), [data-warehouse](<https://devfeed.tech/tags/data-warehouse.md>), [data-warehousing](<https://devfeed.tech/tags/data-warehousing.md>), [databricks](<https://devfeed.tech/tags/databricks.md>), [delta-lake](<https://devfeed.tech/tags/delta-lake.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [modern-data-stack](<https://devfeed.tech/tags/modern-data-stack.md>)

### AI overview

A modern data science stack can bring analytics and machine learning workflows together around governed warehouse data, reducing duplicated processing and the risk of training-serving skew. BigQuery ML and Redshift ML extend SQL with CREATE MODEL workflows for automated or custom models; custom models must be TensorFlow models in BigQuery and deployed through SageMaker for Redshift ML. Both support invoking trained models in SELECT queries and storing predictions in warehouse tables. Snowflake describes an extensibility approach that integrates with tools such as SageMaker and Databricks. Delta Lake takes the reverse path, adding analytics features such as ACID transactions to data lakes as part of a lakehouse architecture. The excerpt presents these as different ways to consolidate data work and pipelines.

### Source excerpt

We evaluate various in-data warehouse machine learning services, and discuss each of their benefits and requirements.

## When Anchors Shift the Target: Advantage-Gated Adaptation for Diffusion Reinforcement Learning

DevFeed: [When Anchors Shift the Target: Advantage-Gated Adaptation for Diffusion Reinforcement Learning](<https://devfeed.tech/articles/when-anchors-shift-the-target-advantage-gated-adaptation-for-diffusion-reinforcement-learning-141832.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.23038>)

Author: Pu Cao, Min Zhou, Ying Shu, Tiezheng Ge, Qing Song, Lu Yang

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>), [Training AI Models](<https://devfeed.tech/topics/training-ai-models.md>)

Tags: [alignment](<https://devfeed.tech/tags/alignment.md>), [coupling](<https://devfeed.tech/tags/coupling.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [diffusion](<https://devfeed.tech/tags/diffusion.md>), [flow-matching](<https://devfeed.tech/tags/flow-matching.md>), [generation](<https://devfeed.tech/tags/generation.md>), [gradient](<https://devfeed.tech/tags/gradient.md>), [guidance](<https://devfeed.tech/tags/guidance.md>), [horizon](<https://devfeed.tech/tags/horizon.md>), [policy](<https://devfeed.tech/tags/policy.md>), [quality](<https://devfeed.tech/tags/quality.md>), [reference](<https://devfeed.tech/tags/reference.md>), [regression](<https://devfeed.tech/tags/regression.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [rendering](<https://devfeed.tech/tags/rendering.md>), [reward](<https://devfeed.tech/tags/reward.md>), [schedules](<https://devfeed.tech/tags/schedules.md>), [standard](<https://devfeed.tech/tags/standard.md>), [tangent](<https://devfeed.tech/tags/tangent.md>)

### AI overview

Advantage-Gated Adaptation (AGA) separates positive target fitting from negative-sample stabilization in diffusion and flow-matching reinforcement learning. Positive samples keep standard regression targets, so their updates vanish at the observed target regardless of the moving policy anchor. Negative samples use a first-order tangent at the moving policy's prediction, retaining the local repulsion direction with a prediction-independent gradient. In the quadratic-anchor family, the paper derives the parameter-free gate ω(a)=max(-a,0). AGA treats the moving copy as a local linearization aid and assigns longer-term behavior preservation to a frozen pretrained reference regularized at its operating classifier-free guidance scale. Across compositional generation, text rendering, and preference alignment, the reported experiments find that shifting positive targets harms visual quality and increases sensitivity to policy-copy schedules, while AGA improves the reward-quality trade-off.

### Source excerpt

arXiv:2611.23038v1 Announce Type: cross Abstract: Forward-process reinforcement learning adapts diffusion and flow-matching models by re-noising rewarded generations into regression targets. Because negative advantages induce concave updates with unbounded repulsive gradients, existing methods stabilize training using cached predictions from a moving policy copy, yet these anchors also alter what positive feedback teaches the model to fit. We formalize this conflict as anchor coupling: after separating frozen-reference regularizers, affine moving-anchor rules cannot eliminate negative curvature while preserving every positive regression target. We propose Advantage-Gated Adaptation (AGA) to decouple these roles. Positive samples retain standard target fitting, whose self-limiting update vanishes at the observed target independently of the moving anchor. Negative samples instead use the first-order tangent of the concave objective at the moving-policy prediction, preserving the local repulsion direction with a prediction-independent gradient. Within the quadratic-anchor family, these requirements uniquely determine the parameter-free gate $\omega(a)=\max(-a,0)$. Because the moving copy provides local linearization rather than retention, AGA assigns long-horizon behavior preservation to a frozen pretrained reference regularized at its operating classifier-free guidance scale, interpreted through an implicit-reward lens. Across compositional generation, text rendering, and preference alignment, our experiments show that displacing positive targets degrades visual quality and heightens sensitivity to policy-copy schedules, whereas AGA consistently improves the reward--quality trade-off.

## Eyes in the Update? Learning Where Rewards Land in Multimodal Reasoning

DevFeed: [Eyes in the Update? Learning Where Rewards Land in Multimodal Reasoning](<https://devfeed.tech/articles/eyes-in-the-update-learning-where-rewards-land-in-multimodal-reasoning-140029.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.22438>)

Author: Lixuan He, Tianzhe Chu, Shulin Tian, Shikang Zheng, Feng Chen, Yi Ma

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>), [post-training](<https://devfeed.tech/topics/post-training.md>), [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Training AI Models](<https://devfeed.tech/topics/training-ai-models.md>), [qwen](<https://devfeed.tech/topics/qwen.md>), [Representation learning](<https://devfeed.tech/topics/representation-learning.md>)

Tags: [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [grounding](<https://devfeed.tech/tags/grounding.md>), [image](<https://devfeed.tech/tags/image.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [multimodal](<https://devfeed.tech/tags/multimodal.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [reasoning](<https://devfeed.tech/tags/reasoning.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [token](<https://devfeed.tech/tags/token.md>), [uncertainty](<https://devfeed.tech/tags/uncertainty.md>), [video](<https://devfeed.tech/tags/video.md>)

### AI overview

Gaze-RL reallocates a trajectory-level reward across response positions using the model's internal cross-modal geometry and local predictive uncertainty. It gives greater update weight to positions that are visually anchored and decision-bearing while keeping weights mean-one, preserving each rollout's total credit. The method requires no process supervision, auxiliary perception module, or architectural change. The abstract reports improvements for Qwen2.5-VL and InternVL3.5 across image and video benchmarks; it provides no benchmark details or quantitative results.

### Source excerpt

arXiv:2611.22438v1 Announce Type: cross Abstract: Multimodal post-training has moved from broad instruction tuning toward reinforcement learning for image and video reasoning, where outcome rewards can improve final-answer accuracy. Yet these rewards are assigned to complete generated trajectories, while visual evidence is used only at sparse and uneven positions inside the response. Recent work has begun to make multimodal RL more token-sensitive through grounding signals or external token-level weighting. We ask a more specific question: can the allocation of the same trajectory-level reward be inferred from the model's own multimodal computation, rather than from an auxiliary annotator, perception module, or process label? We propose Gaze-RL, a representation-driven policy optimization method that redistributes completion-level advantage through a latent reward landing field. The field is computed from internal cross-modal geometry and local predictive uncertainty, assigning larger update weight to positions that are both visually anchored and decision-bearing while remaining mean-one, so the total credit of each rollout is preserved and only its positionwise allocation changes. Gaze-RL requires no process supervision, auxiliary perception module, or architectural modification, and improves both Qwen2.5-VL and InternVL3.5 across a broad range of image and video benchmarks.

## Agentic In-Context Learning: What Algorithms Can Transformers Learn in Closed-Loop Interaction?

DevFeed: [Agentic In-Context Learning: What Algorithms Can Transformers Learn in Closed-Loop Interaction?](<https://devfeed.tech/articles/agentic-in-context-learning-what-algorithms-can-transformers-learn-in-closed-loop-interaction-139838.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.23051>)

Author: Shiguang Wu, Yaqing Wang, Quanming Yao

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>), [post-training](<https://devfeed.tech/topics/post-training.md>), [Training AI Models](<https://devfeed.tech/topics/training-ai-models.md>), [Transformers](<https://devfeed.tech/topics/transformers.md>)

Tags: [agentic](<https://devfeed.tech/tags/agentic.md>), [agents](<https://devfeed.tech/tags/agents.md>), [algorithms](<https://devfeed.tech/tags/algorithms.md>), [alignment](<https://devfeed.tech/tags/alignment.md>), [bandits](<https://devfeed.tech/tags/bandits.md>), [behavior](<https://devfeed.tech/tags/behavior.md>), [context](<https://devfeed.tech/tags/context.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [decision](<https://devfeed.tech/tags/decision.md>), [distillation](<https://devfeed.tech/tags/distillation.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [interaction](<https://devfeed.tech/tags/interaction.md>), [learn](<https://devfeed.tech/tags/learn.md>), [learning](<https://devfeed.tech/tags/learning.md>), [online](<https://devfeed.tech/tags/online.md>), [post-training](<https://devfeed.tech/tags/post-training.md>), [process](<https://devfeed.tech/tags/process.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [reward](<https://devfeed.tech/tags/reward.md>), [rl](<https://devfeed.tech/tags/rl.md>), [transformers](<https://devfeed.tech/tags/transformers.md>)

### AI overview

The paper frames interactive learning by language model agents as Agentic In-Context Learning: an agent's actions shape what evidence enters its context. It models learners through their state, update, and decision rule, covering active learning, bandits, and online RL. The authors construct causal Transformers that approximate these algorithms over a finite horizon and bound the resulting return loss, and characterize when process-reward optimization preserves outcome-maximizing behavior. Distillation experiments show Transformers can execute three representative recursions. Training with outcome or process rewards, without oracle action or state targets, produces distinct policies; better alignment between process supervision and outcome-optimal action preferences shifts decisions accordingly. The abstract reports that these differences persist across textual paraphrases in a pretrained language model.

### Source excerpt

arXiv:2610.23051v2 Announce Type: replace Abstract: The learning algorithm view of in-context learning (ICL) studies how fixed-parameter Transformers learn from supplied examples. Language model agents extend this setting through interaction: their actions determine what evidence enters context next. We call this regime Agentic In-Context Learning (Agentic ICL). Existing studies largely depend on externally supplied contexts or reward-bearing interaction histories, while work on general tool-using agents lacks a learning algorithm view. We represent an interactive learner by its state, update, and decision rule, covering active learning, bandits, and online reinforcement learning (RL). For expressivity, we construct causal Transformers that approximate these algorithms over a finite horizon, and bound the resulting loss in return. For training, we characterize when optimizing process rewards preserves behavior that maximizes the final outcome. Distillation experiments show that Transformers can learn to execute three representative recursions. We then train models from outcome or process rewards without oracle action or state targets, as in RL post-training. Different rewards yield distinct decision policies. Improving the alignment of process supervision with outcome-optimal action preferences shifts learned decisions accordingly. Such behavioral differences persist in a pretrained language model across textual paraphrases.

## GP--RL--MPC: Uncertainty-Aware Reinforcement Learning-Augmented Model Predictive Control with Gaussian Process Postdecision State-Value Functions

DevFeed: [GP--RL--MPC: Uncertainty-Aware Reinforcement Learning-Augmented Model Predictive Control with Gaussian Process Postdecision State-Value Functions](<https://devfeed.tech/articles/gp-rl-mpc-uncertainty-aware-reinforcement-learning-augmented-model-predictive-control-with-gaussian-process-postdecision-state-value-functions-141722.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.22038>)

Author: Jon Li, Jay H. Lee

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>), [Training AI Models](<https://devfeed.tech/topics/training-ai-models.md>), [Predictive AI](<https://devfeed.tech/topics/predictive-ai.md>)

Tags: [control](<https://devfeed.tech/tags/control.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-sy](<https://devfeed.tech/tags/cs-sy.md>), [eess-sy](<https://devfeed.tech/tags/eess-sy.md>), [gaussian](<https://devfeed.tech/tags/gaussian.md>), [learning](<https://devfeed.tech/tags/learning.md>), [model](<https://devfeed.tech/tags/model.md>), [mpc](<https://devfeed.tech/tags/mpc.md>), [process](<https://devfeed.tech/tags/process.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [state](<https://devfeed.tech/tags/state.md>), [uncertainty](<https://devfeed.tech/tags/uncertainty.md>), [value](<https://devfeed.tech/tags/value.md>)

### AI overview

GP--RL--MPC uses a learned Gaussian-process critic as the MPC terminal penalty to represent costs beyond the prediction horizon. It learns from limited, noisy rollouts through optimistic approximate policy iteration; soft updates are used to mitigate policy chattering, and a tunable risk coefficient weights epistemic uncertainty in the terminal cost. Its postdecision-state formulation shifts disturbance averaging offline to account for aleatoric uncertainty after the terminal transition. A one-step horizon recovers greedy value-space RL when objectives and admissible actions match. For scaling, the method fits GP hyperparameters by exact marginal likelihood and uses a sparse DTC posterior with pivoted-Cholesky-selected inducing points. In a disturbed CSTR, an appropriately chosen risk coefficient and policy screening reduced SSE by an average 80.52% versus a matched baseline MPC across five random seeds (sample SD 3.48%).

### Source excerpt

arXiv:2611.22038v1 Announce Type: new Abstract: We propose Gaussian Process Reinforcement Learning-Augmented Model Predictive Control (GP--RL--MPC), which uses a learned critic as the MPC terminal penalty to account for costs beyond the prediction horizon. Gaussian process regression (GPR) enables data-efficient critic learning from limited, noisy rollouts within optimistic approximate policy iteration (OAPI), with soft updates mitigating policy chattering. A tunable risk coefficient weights GP epistemic uncertainty in the terminal cost, regulating sensitivity to uncertain critic predictions. The postdecision state formulation shifts disturbance averaging offline to account for aleatoric uncertainty from the terminal transition onward. Under matching objectives and admissible actions, a one-step horizon recovers greedy value-space RL. After fitting GP hyperparameters by exact marginal likelihood, we construct a sparse deterministic training conditional (DTC) posterior using inducing points selected by pivoted Cholesky (PC), reducing online evaluation costs as the training dataset grows. In a continuous stirred-tank reactor (CSTR) subject to exogenous disturbances, GP--RL--MPC with an appropriately selected risk coefficient and policy screening reduces the sum of squared errors (SSE) by an average of 80.52% relative to an otherwise matched baseline MPC across five random seeds (sample standard deviation: 3.48%).

## KneeFS-Net: A Lightweight Deep Learning Model for Knee Osteoarthritis Severity Assessment with Cross-Domain External Validation

DevFeed: [KneeFS-Net: A Lightweight Deep Learning Model for Knee Osteoarthritis Severity Assessment with Cross-Domain External Validation](<https://devfeed.tech/articles/kneefs-net-a-lightweight-deep-learning-model-for-knee-osteoarthritis-severity-assessment-with-cross-domain-external-validation-138972.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.19988>)

Author: Shafqat Alam, Galib Anjum Talukder Mahi, Yeasir Rabbani Tanvir, Md. Ibrahim, Md. Shahed Hossain, Shashank Ghimire, Mahmud Hasan Mostofa Kamal, ASM Shahidul Hossain, Taufiq Hasan

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Training AI Models](<https://devfeed.tech/topics/training-ai-models.md>), [Image processing](<https://devfeed.tech/topics/image-processing.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [Inference](<https://devfeed.tech/topics/inference.md>)

Tags: [assessment](<https://devfeed.tech/tags/assessment.md>), [cross-validation](<https://devfeed.tech/tags/cross-validation.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [deep-learning](<https://devfeed.tech/tags/deep-learning.md>), [domain](<https://devfeed.tech/tags/domain.md>), [external](<https://devfeed.tech/tags/external.md>), [features](<https://devfeed.tech/tags/features.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [lightweight](<https://devfeed.tech/tags/lightweight.md>), [model](<https://devfeed.tech/tags/model.md>), [pretraining](<https://devfeed.tech/tags/pretraining.md>), [validation](<https://devfeed.tech/tags/validation.md>)

### AI overview

KneeFS-Net is a compact model for grading knee osteoarthritis on radiographs using the Kellgren-Lawrence scale. It combines spatial, Fourier and state-space features with landmark-guided attention, using 0.8 million parameters and 0.33 GFLOPs. On the OAI test set it achieved QWK 0.863, MAE 0.325 and 97.2% within-one-grade accuracy, comparable to models with 9-35 times more parameters. Cross-population performance varied: without adaptation, the OAI-trained model reached QWK 0.803 on a single-center Bangladeshi hospital cohort, but QWK 0.656 on multi-center teleradiology images. Self-supervised pretraining on 6,092 unlabeled radiographs followed by joint fine-tuning achieved QWK 0.823 and 99.7% within-one-grade accuracy in cross-validation. The abstract suggests adaptation with limited annotation may help, while the excerpt does not provide further methodological or validation details.

### Source excerpt

arXiv:2611.19988v1 Announce Type: cross Abstract: Knee osteoarthritis (OA) grading from radiographs using the Kellgren-Lawrence (KL) scale requires expertise that may be scarce. We present KneeFS-Net, a lightweight ordinal grader (0.8 million parameters; 0.33 GFLOPs) combining spatial, Fourier and state-space features with landmark-guided attention. On the Osteoarthritis Initiative (OAI) test set, KneeFS-Net achieved a quadratic weighted kappa (QWK) of 0.863, mean absolute error (MAE) of 0.325 and 97.2% within-one-grade accuracy, comparable to models with 9-35-fold more parameters. We evaluated cross-population generalization on two Bangladeshi cohorts, a single-center hospital cohort and a multi-center teleradiology cohort. Without further training or adaptation, the OAI-trained model achieved QWK 0.803, MAE 0.339 and 98.2% within-one-grade accuracy on its hospital cohort, declining to QWK 0.656 and MAE 0.522 on multi-center teleradiology images. Self-supervised pretraining on 6,092 unlabeled radiographs followed by joint fine-tuning achieved QWK 0.823, MAE 0.329 and 99.7% within-one-grade accuracy in cross-validation. These findings demonstrate accurate, efficient KL grading and potential adaptation to new populations with limited annotation.

## Sample-Efficient Ensemble Reinforcement Learning at Scale

DevFeed: [Sample-Efficient Ensemble Reinforcement Learning at Scale](<https://devfeed.tech/articles/sample-efficient-ensemble-reinforcement-learning-at-scale-138920.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.19733>)

Author: Yoav Feigenbaum, Dmitrii Krylov, Roy Fox

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Training AI Models](<https://devfeed.tech/topics/training-ai-models.md>)

Tags: [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [sample](<https://devfeed.tech/tags/sample.md>), [scale](<https://devfeed.tech/tags/scale.md>)

### AI overview

MeanQ-BBF combines MeanQ ensembling with the larger-scale BBF architecture to improve sample-efficient reinforcement learning. On the Atari 100K benchmark, the authors report that simply increasing BBF network size degrades performance, while MeanQ-BBF uses the added capacity to improve performance over BBF. It organizes added parameters into largely independent learners to keep computation efficient. The abstract says the paper also discusses the performance-cost tradeoff, but provides no specific cost figures or detailed results.

### Source excerpt

arXiv:2611.19733v1 Announce Type: new Abstract: Ensemble methods such as MeanQ have been shown to improve the sample efficiency of relatively small model-free deep reinforcement learning (RL) architectures, including DQN and similarly sized variants such as Rainbow. However, to our knowledge, the effects of ensembling on large-scale architectures such as Bigger Better Faster (BBF) have not been studied. In this work, we present MeanQ-BBF, a hybrid model-free algorithm that combines the principles of MeanQ and BBF. Through statistically rigorous evaluation on the Atari 100K benchmark, we show that in contrast to naive scaling of the network size of BBF, which degrades performance, MeanQ-BBF is able to leverage the increase in model capacity to improve performance over BBF. Moreover, MeanQ-BBF does this in a computationally efficient way by organizing the added parameters into largely independent learners. Finally, we discuss the performance-cost tradeoff in the context of practical sample-efficient RL.

## Multi-CONDOR: Learning Multistable Motion Primitives from Demonstration

DevFeed: [Multi-CONDOR: Learning Multistable Motion Primitives from Demonstration](<https://devfeed.tech/articles/multi-condor-learning-multistable-motion-primitives-from-demonstration-137649.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.21059>)

Author: Gabriele Giuseppe Di Marzo, Mariano Ram\\'irez Montero, Maximilian St\\"olzle, Rodrigo P\\'erez-Dattari, Cosimo Della Santina

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Robot Manipulation](<https://devfeed.tech/topics/robot-manipulation.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Training AI Models](<https://devfeed.tech/topics/training-ai-models.md>)

Tags: [alternatives](<https://devfeed.tech/tags/alternatives.md>), [analysis](<https://devfeed.tech/tags/analysis.md>), [arm](<https://devfeed.tech/tags/arm.md>), [autonomous](<https://devfeed.tech/tags/autonomous.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [challenge](<https://devfeed.tech/tags/challenge.md>), [container](<https://devfeed.tech/tags/container.md>), [continuous](<https://devfeed.tech/tags/continuous.md>), [contour](<https://devfeed.tech/tags/contour.md>), [convergence](<https://devfeed.tech/tags/convergence.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-ro](<https://devfeed.tech/tags/cs-ro.md>), [either](<https://devfeed.tech/tags/either.md>), [encoding](<https://devfeed.tech/tags/encoding.md>), [examples](<https://devfeed.tech/tags/examples.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [flexibility](<https://devfeed.tech/tags/flexibility.md>), [framework](<https://devfeed.tech/tags/framework.md>), [goals](<https://devfeed.tech/tags/goals.md>), [imitation](<https://devfeed.tech/tags/imitation.md>), [learning](<https://devfeed.tech/tags/learning.md>), [motion](<https://devfeed.tech/tags/motion.md>)

### AI overview

Multi-CONDOR learns reactive robot motion primitives that can converge to multiple discrete goals or a continuous goal contour. It combines imitation learning with losses that align learned dynamics to latent reference systems encoding the desired attraction structure, and includes a Lyapunov-based convergence analysis. In simulation, it reduced trajectory RMSE by about 72% versus separately learned single-goal primitives selected by a state machine. For tasks with two and three goals, respectively, the share of evaluated initial states ending outside goal tolerance fell from StableNODE's 0.31% and 7.29% to 0% in both cases. Franka Emika arm experiments demonstrated grasping a container by either of its two handles from demonstrations. The evidence includes simulated benchmarks, qualitative continuous-contour examples, and robotic experiments.

### Source excerpt

arXiv:2611.21059v1 Announce Type: cross Abstract: Dynamical-system-based motion policies provide a framework for learning reactive robot motions with stability and convergence properties. Many manipulation tasks, however, admit several distinct goals or a continuous set of acceptable outcomes, requiring motion primitives that preserve these alternatives. Accommodating both goal structures within a unified framework with continuous, autonomous dynamics remains an open challenge. We propose a CONDOR-based framework, multi-CONDOR, for learning motion primitives with either multiple discrete attractors or a continuous attracting goal contour. The approach combines imitation learning with losses that encourage agreement between the learned task dynamics and latent reference systems encoding the desired attraction structure. We provide a Lyapunov-based analysis of the convergence properties of the learned motion dynamics. We evaluate the framework on simulated benchmarks with discrete goals, qualitative examples with continuous goal contours, and robotic grasping experiments. In simulation, multi-CONDOR reduces trajectory RMSE by approximately 72% compared with separately learned single-goal motion primitives selected by a state machine. For tasks with two and three goals, respectively, it reduces the fraction of evaluated initial states ending outside the goal tolerance from StableNODE's 0.31% and 7.29% to 0% in both settings. Experiments with a Franka Emika robot arm demonstrate successful grasping of a container by either of its two handles using motions learned from demonstrations. Together, these results show multi-CONDOR combines motion reproduction with the flexibility to converge to alternative valid outcomes.

## Microfitting: Continual Learning via Cached Test-Time Training

DevFeed: [Microfitting: Continual Learning via Cached Test-Time Training](<https://devfeed.tech/articles/microfitting-continual-learning-via-cached-test-time-training-137553.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.20258>)

Author: Nick Ryan, Peter A. Devine, Ben Sirb, Hannah Sansford, Alex Chiocchi

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Training AI Models](<https://devfeed.tech/topics/training-ai-models.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Inference](<https://devfeed.tech/topics/inference.md>)

Tags: [adapter](<https://devfeed.tech/tags/adapter.md>), [continual-learning](<https://devfeed.tech/tags/continual-learning.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [private](<https://devfeed.tech/tags/private.md>), [test](<https://devfeed.tech/tags/test.md>), [training](<https://devfeed.tech/tags/training.md>)

### AI overview

Microfitting specializes a frozen language model to private documents by training and caching a small adapter for each document. Each adapter is trained offline once on synthetic questions aimed at demonstrated model shortcomings, then reused when that document is queried. Keeping adapters isolated is intended to avoid overwriting the base model or prior document adaptations; the process for generating training data also revises its recipe based on previous outcomes and fresh workload examples. On DeepSeek V4 Flash, the abstract reports a 38.2% relative accuracy improvement over the base model on Enterprise Document Operations, an open-book Q&A benchmark using real-world enterprise documents. It describes adapting overnight on local hardware, but the supplied abstract does not provide further implementation details or comparative results.

### Source excerpt

arXiv:2611.20258v1 Announce Type: cross Abstract: Continual learning is a promising approach to specialize language models to an organization's private data. However, current methods exhibit a number of drawbacks including catastrophic forgetting, where new training updates overwrite prior knowledge. We present microfitting, which trains and caches a small adapter for each document. Each adapter is trained once, offline, on synthetic questions targeted at demonstrated base-model shortcomings and reused whenever that document is queried. This method borrows from test-time learning literature and sidesteps the problem of continual learning through parameter isolation, allowing the system to specialize on new data without overwriting prior knowledge. The process that generates adapter training data also improves over time, revising its recipe from previous outcomes and fresh examples of the target workload. On DeepSeek V4 Flash, microfitting substantially improves accuracy over the base model. In Enterprise Document Operations (EDO), an open-book Q&A environment over real-world enterprise documents, our method achieves a 38.2% relative improvement over base-model performance. Microfitting makes continual specialization practical and cost-efficient: a frozen model can adapt to an evolving private corpus overnight on local hardware, without modifying the base model or previously cached adapters.

## Geometrically Disentangling Concept Learning from the Language Modeling Loss

DevFeed: [Geometrically Disentangling Concept Learning from the Language Modeling Loss](<https://devfeed.tech/articles/geometrically-disentangling-concept-learning-from-the-language-modeling-loss-138807.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.18933>)

Author: Yupei Wang, Neil Mallinar, Mikhail Belkin, Alex Warstadt

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Language models](<https://devfeed.tech/topics/language-models.md>), [Training AI Models](<https://devfeed.tech/topics/training-ai-models.md>), [Representation learning](<https://devfeed.tech/topics/representation-learning.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>)

Tags: [alignment](<https://devfeed.tech/tags/alignment.md>), [concept-learning](<https://devfeed.tech/tags/concept-learning.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [energy](<https://devfeed.tech/tags/energy.md>), [interpretability](<https://devfeed.tech/tags/interpretability.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [learning](<https://devfeed.tech/tags/learning.md>), [lexical](<https://devfeed.tech/tags/lexical.md>), [modeling](<https://devfeed.tech/tags/modeling.md>), [next-token-prediction](<https://devfeed.tech/tags/next-token-prediction.md>), [pretraining](<https://devfeed.tech/tags/pretraining.md>), [semantic](<https://devfeed.tech/tags/semantic.md>), [training](<https://devfeed.tech/tags/training.md>)

### AI overview

The paper introduces Concept Energy, which measures how strongly a concept's representation subspace aligns with directions in which the next-token-prediction loss is sensitive, captured by the average gradient outer product (AGOP). Across 19 checkpoints of Pythia-410M and Pythia-1B and 12 lexical, syntactic, and semantic concepts, the first Concept Energy peak generally occurs at or before the steepest rise in probe decodability. Unlike probe performance, which usually increases monotonically, Concept Energy can show sustained or repeated engagement during pretraining, suggesting concept learning may involve continual reinforcement or multiple learning periods. The results indicate that probe performance alone does not reveal the varied ways the language-modeling loss relates to concept development.

### Source excerpt

arXiv:2611.18933v1 Announce Type: new Abstract: Language models acquire a vast array of concepts and abilities from next-token prediction alone. We introduce an interpretability framework that exposes how this single loss relates to the development of different concepts at different points during pretraining. We propose Concept Energy, a measure of the concept-specific alignment between a concept subspace and the average gradient outer product (AGOP) of the next-token-prediction loss. The concept subspace captures where the concept currently lives in the model's representation; AGOP captures the directions to which the loss is most sensitive. Across 19 checkpoints of Pythia-410M and -1B and 12 lexical, syntactic, and semantic concepts, we find that the first peak in Concept Energy generally occurs at or before the steepest increase in probe decodability. Furthermore, whereas probe performance usually increases monotonically throughout training, Concept Energy is not confined to a single peak; some concepts exhibit sustained or repeated engagement across pretraining, suggesting that concept learning may require continual reinforcement or occur at multiple points. Our results indicate that the loss contributes to concept learning in a myriad of ways that simple probe performance alone cannot reveal.

## Optimistic Verifiers Get Hacked: Reward Hacking Under an Exact Process Oracle

DevFeed: [Optimistic Verifiers Get Hacked: Reward Hacking Under an Exact Process Oracle](<https://devfeed.tech/articles/optimistic-verifiers-get-hacked-reward-hacking-under-an-exact-process-oracle-137713.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.21538>)

Author: Arunkumar V, Manoranjan Gandhudi, Gangadharan G. R., Arun Prakash, S. Senthilkumar

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [rlvr](<https://devfeed.tech/topics/rlvr.md>), [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>), [Training AI Models](<https://devfeed.tech/topics/training-ai-models.md>), [audit](<https://devfeed.tech/topics/audit.md>)

Tags: [audit](<https://devfeed.tech/tags/audit.md>), [authorization](<https://devfeed.tech/tags/authorization.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [hacked](<https://devfeed.tech/tags/hacked.md>), [hacking](<https://devfeed.tech/tags/hacking.md>), [instrumentation](<https://devfeed.tech/tags/instrumentation.md>), [llm](<https://devfeed.tech/tags/llm.md>), [magnitude](<https://devfeed.tech/tags/magnitude.md>), [noise](<https://devfeed.tech/tags/noise.md>), [opa](<https://devfeed.tech/tags/opa.md>), [oracle](<https://devfeed.tech/tags/oracle.md>), [padding](<https://devfeed.tech/tags/padding.md>), [policies](<https://devfeed.tech/tags/policies.md>), [process](<https://devfeed.tech/tags/process.md>), [proxy](<https://devfeed.tech/tags/proxy.md>), [ranking](<https://devfeed.tech/tags/ranking.md>), [recall](<https://devfeed.tech/tags/recall.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [reward](<https://devfeed.tech/tags/reward.md>), [testbed](<https://devfeed.tech/tags/testbed.md>), [train](<https://devfeed.tech/tags/train.md>)

### AI overview

The study tests reward hacking in verifiable-reward training using executable authorization policies across 124,416 contexts, where an oracle can identify every minimal sufficient reason for each decision. Training against seven degraded verifiers revealed four patterns: optimistic sufficiency proxies encouraged padded explanations, raising proxy scores to 0.940 while held-out true faithfulness fell from 0.731 to 0.667; outcome-only verifiers saturated, extraction noise diluted the signal, and ranking-preserving gaps caused no harm. A model-free audit of selection error and gap sign, plus a short saturation pilot, predicted the observed pattern. The patterns recurred across nine configurations; sufficiency damage was positive in all 10 replication cells and exceeded two standard errors in seven. However, its magnitude did not replicate. The authors release the testbed, training instrumentation, and 14 public policies converted to executable oracles.

### Source excerpt

arXiv:2611.21538v1 Announce Type: cross Abstract: Reinforcement learning from verifiable rewards inherits whatever its verifier gets wrong. Outside mathematics and code the verifier is a proxy, a rubric, a learned judge or a hand-written checker, and its distance from the true objective is never measured, so a rising reward cannot be told apart from a hack. We build a setting where the true reward is exactly computable at the process level: executable authorization policies over 124,416 contexts admit an oracle that lists every minimal sufficient reason for every decision. We compile these policies into exact process rewards, degrade the verifier along seven realistic axes (outcome-only, sufficiency, recall-only, extraction noise, coarse taxonomy, stale policy, LLM judge), and train against each while the exact verifier scores every rollout. Four phenotypes appear. Optimistic verifiers are hacked by padding: the sufficiency proxy ends at 0.940 while true faithfulness falls from 0.731 to 0.667 on 96 held-out episodes. Outcome-only verifiers saturate and stop teaching, noise dilutes, and ranking-preserving gaps do no harm. A model-free audit of the verifier, its selection error and gap sign, together with a short saturation pilot, anticipates which phenotype a run will produce. Across nine configurations spanning two frameworks, two model vendors, two adapter sizes, three precisions, two initialisations and two seeds, the phenotypes recur, sufficiency damage is positive in all 10 replication cells and beyond two standard errors in 7, and optimistic gaps remain the steeper branch. The damage magnitude does not replicate, and five pre-registered attempts to explain it are reported as they fell. We release the testbed, the training instrumentation, and 14 real public policies (Cedar, OPA, Gatekeeper) ported to executable oracles.

## Learning the language of machine learning

DevFeed: [Learning the language of machine learning](<https://devfeed.tech/articles/learning-the-language-of-machine-learning-138104.md>)

Original publisher: [Read original article](<https://news.mit.edu/2026/learning-language-of-machine-learning-jacob-andreas-1117>)

Author: Adam Zewe | MIT News

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [MIT AI News](<https://devfeed.tech/sources/mit-ai-news.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Neural Network](<https://devfeed.tech/topics/neural-network.md>), [Training AI Models](<https://devfeed.tech/topics/training-ai-models.md>), [ai safety](<https://devfeed.tech/topics/ai-safety.md>)

Tags: [2019](<https://devfeed.tech/tags/2019.md>), [about](<https://devfeed.tech/tags/about.md>), [academic](<https://devfeed.tech/tags/academic.md>), [adapting](<https://devfeed.tech/tags/adapting.md>), [addition](<https://devfeed.tech/tags/addition.md>), [advanced](<https://devfeed.tech/tags/advanced.md>), [after](<https://devfeed.tech/tags/after.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [associate](<https://devfeed.tech/tags/associate.md>), [can](<https://devfeed.tech/tags/can.md>), [computer-science-and-artificial-intelligence-laboratory-csail](<https://devfeed.tech/tags/computer-science-and-artificial-intelligence-laboratory-csail.md>), [computer-science-and-technology](<https://devfeed.tech/tags/computer-science-and-technology.md>), [data](<https://devfeed.tech/tags/data.md>), [effectively](<https://devfeed.tech/tags/effectively.md>), [electrical-engineering-and-computer-science-eecs](<https://devfeed.tech/tags/electrical-engineering-and-computer-science-eecs.md>), [faculty](<https://devfeed.tech/tags/faculty.md>), [generative-ai](<https://devfeed.tech/tags/generative-ai.md>), [humans](<https://devfeed.tech/tags/humans.md>), [intelligent-systems](<https://devfeed.tech/tags/intelligent-systems.md>), [jacob-andreas](<https://devfeed.tech/tags/jacob-andreas.md>), [language](<https://devfeed.tech/tags/language.md>), [language-learning](<https://devfeed.tech/tags/language-learning.md>), [large-language-models](<https://devfeed.tech/tags/large-language-models.md>), [learning](<https://devfeed.tech/tags/learning.md>), [machine](<https://devfeed.tech/tags/machine.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [mit-ai-faculty](<https://devfeed.tech/tags/mit-ai-faculty.md>), [mit-schwarzman-college-of-computing](<https://devfeed.tech/tags/mit-schwarzman-college-of-computing.md>), [natural-language-processing](<https://devfeed.tech/tags/natural-language-processing.md>), [profile](<https://devfeed.tech/tags/profile.md>), [research](<https://devfeed.tech/tags/research.md>), [school-of-engineering](<https://devfeed.tech/tags/school-of-engineering.md>), [systems](<https://devfeed.tech/tags/systems.md>), [with](<https://devfeed.tech/tags/with.md>)

### AI overview

Jacob Andreas's group develops ways for AI models to describe their learned representations in human-readable language, helping researchers anticipate model behavior in real-world and high-stakes settings. One approach trains language models to explain other neural networks by assigning natural-language descriptions to processing nodes, such as a node that detects patches of grass near animals. The group also studies how models learn and designs training algorithms that predict how particular datasets or examples may affect behavior. The article identifies a limitation: researchers still lack a good scientific theory for predicting what training on a given dataset will produce. Andreas sees interpretability as increasingly important for deploying advanced systems safely, including models trained on scientific data that researchers may not yet understand.

### Source excerpt

By studying the computational foundations of language learning, Associate Professor Jacob Andreas builds intelligent systems that can communicate effectively with humans.

## Differentiable Exact Learning

DevFeed: [Differentiable Exact Learning](<https://devfeed.tech/articles/differentiable-exact-learning-137707.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.21486>)

Author: Hristo Papazov, Francesco D'Angelo, Nicolas Flammarion

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Finite-state machine](<https://devfeed.tech/topics/finite-state-machine.md>), [Neural Network](<https://devfeed.tech/topics/neural-network.md>), [generalization in machine learning](<https://devfeed.tech/topics/generalization-in-machine-learning.md>), [Training AI Models](<https://devfeed.tech/topics/training-ai-models.md>)

Tags: [algorithms](<https://devfeed.tech/tags/algorithms.md>), [arithmetic](<https://devfeed.tech/tags/arithmetic.md>), [controller](<https://devfeed.tech/tags/controller.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-fl](<https://devfeed.tech/tags/cs-fl.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [environment](<https://devfeed.tech/tags/environment.md>), [finite-state](<https://devfeed.tech/tags/finite-state.md>), [learning](<https://devfeed.tech/tags/learning.md>), [machine](<https://devfeed.tech/tags/machine.md>), [protocol](<https://devfeed.tech/tags/protocol.md>), [replicate](<https://devfeed.tech/tags/replicate.md>), [simulation](<https://devfeed.tech/tags/simulation.md>), [stat-ml](<https://devfeed.tech/tags/stat-ml.md>), [state](<https://devfeed.tech/tags/state.md>), [target](<https://devfeed.tech/tags/target.md>), [traces](<https://devfeed.tech/tags/traces.md>), [tracking](<https://devfeed.tech/tags/tracking.md>), [train](<https://devfeed.tech/tags/train.md>), [training](<https://devfeed.tech/tags/training.md>), [universality](<https://devfeed.tech/tags/universality.md>)

### AI overview

The paper's approach combines a state-tracking controller, an unbounded external environment, and policy-trajectory observations (expert traces showing observations and actions but hiding internal states) to learn controllers that generalize to much longer tasks. Its Differentiable Finite-State Transducers (DFSTs) use multilinear updates, allow log-parallel training, and can emulate arbitrary finite-state controllers. Compact DFSTs trained on tiny datasets show extreme length generalization on arithmetic and universal simulation; the authors report similar results with RNNs and GRUs. They also extract interpretable discrete controllers from trained DFSTs and prove them equivalent to the target algorithms. For Softmax DFSTs, taking the argmax of transition probabilities yields controllers that provably perform binary arithmetic and universal Turing-machine simulation exactly. Exact-learning proofs for neural controllers require assumptions, which the authors say are empirically supported.

### Source excerpt

arXiv:2611.21486v1 Announce Type: cross Abstract: We introduce a framework for differentiable exact learning that couples an unbounded external environment to a state-tracking neural controller, which we train on policy-trajectory observations (PTOs): expert behavior traces that expose observations and actions while hiding internal states. In addition to the learning framework, we introduce Differentiable Finite-State Transducers (DFSTs) which use multilinear updates, support log-parallel training, and emulate arbitrary finite-state controllers, yielding computational universality through environmental interaction. From tiny training datasets, compact DFSTs achieve extreme length generalization on arithmetic and universal-simulation tasks. We replicate the success of DFSTs on RNNs and GRUs, thereby supporting a broader learning recipe: State-Tracking Controller + Unbounded External Environment + PTOs = Extreme Length Generalization. Beyond length generalization, our metric-extraction protocol recovers interpretable discrete controllers from trained DFSTs, which we prove equivalent to the target arithmetic and universal-simulation algorithms. Furthermore, for our Softmax DFST models, simply taking $\mathrm{argmax}$ over the transition probabilities yields discrete controllers which provably exactly perform universal Turing-machine simulation and binary arithmetic. Proving exact learning for the neural controllers requires assumptions, which we empirically support. We make our code, training recipes, model weights, and proof checkers publicly available at https://github.com/fanagor/DEL.

[Next page](<https://devfeed.tech/topics/training-ai-models.md?cursor=WyIyMDI2LTExLTE3VDA1OjAwOjAwKzAwOjAwIiwgIjE0YjQ5NDg1LTJmMjUtNGFkYS1iOTgzLWJkZDNkMWVlYTc2ZiJd>)