# arXiv: Machine Learning

Research papers on machine learning algorithms, neural networks, model training, optimization, and learning systems.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## Some Memories Must Fade: Persistence and Decay of User Memories in Model Parameters

DevFeed: [Some Memories Must Fade: Persistence and Decay of User Memories in Model Parameters](<https://devfeed.tech/articles/some-memories-must-fade-persistence-and-decay-of-user-memories-in-model-parameters-145026.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.25235>)

Author: Sanidhya Vijayvargiya, Rahul Lokesh

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [AI Context Management](<https://devfeed.tech/topics/ai-context-management.md>), [lora](<https://devfeed.tech/topics/lora.md>), [Retrieval Augmented Generation (RAG)](<https://devfeed.tech/topics/retrieval-augmented-generation-rag.md>), [LLM evaluation / benchmarking](<https://devfeed.tech/topics/llm-evaluation-benchmarking.md>)

Tags: [adapter](<https://devfeed.tech/tags/adapter.md>), [behavioral-shifts](<https://devfeed.tech/tags/behavioral-shifts.md>), [capacity](<https://devfeed.tech/tags/capacity.md>), [cl](<https://devfeed.tech/tags/cl.md>), [continual-learning](<https://devfeed.tech/tags/continual-learning.md>), [controller](<https://devfeed.tech/tags/controller.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [dataset](<https://devfeed.tech/tags/dataset.md>), [horizon](<https://devfeed.tech/tags/horizon.md>), [knowledge](<https://devfeed.tech/tags/knowledge.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [large-language-models](<https://devfeed.tech/tags/large-language-models.md>), [lifecycle](<https://devfeed.tech/tags/lifecycle.md>), [lora](<https://devfeed.tech/tags/lora.md>), [manage](<https://devfeed.tech/tags/manage.md>), [management](<https://devfeed.tech/tags/management.md>), [memory](<https://devfeed.tech/tags/memory.md>), [memory-management](<https://devfeed.tech/tags/memory-management.md>), [model-weights](<https://devfeed.tech/tags/model-weights.md>), [persistence](<https://devfeed.tech/tags/persistence.md>), [user](<https://devfeed.tech/tags/user.md>)

### AI overview

The paper proposes managing user memories directly in model weights, with a fixed-capacity LoRA adapter for storing facts and a learned lightweight controller for deciding what to retain, update, reinforce, or let fade. The adapter uses k-hot rank combinations intended to reduce catastrophic interference and supports independently controlled temporal decay. Evaluation on controlled long-horizon user trajectories tests recall while facts remain valid and abstention after they expire. The learned policy reaches 86% of oracle performance by balancing recall and abstention; the authors report that continual-learning and retrieval-augmented-generation baselines tend to default mainly to one behavior. These findings are from the described dataset and evaluation.

### Source excerpt

arXiv:2611.25235v1 Announce Type: new Abstract: Parametric memory allows Large Language Models to internalize knowledge directly into their weights, offering a potentially powerful alternative to retrieval-based memory (RAG). However, existing approaches to parametric memory are difficult to deploy as they largely mirror the Continual Learning (CL) objective of retaining information indefinitely. This is neither sustainable nor sufficient for a memory system, which must govern knowledge across its lifecycle by retaining stable facts, reinforcing recurring observations, updating changed beliefs, and allowing transient information to fade. We propose a system for parametric memory management consisting of a memory substrate (handling information storage and forgetting) and a learned policy (what to remember, how deeply, and for how long). Our substrate is a fixed-capacity LoRA adapter that consists of k-hot rank combinations which store facts without catastrophic interference and maintain an independently controlled temporal decay, enabling outdated information to fade while long-term facts persist. On this substrate, we train a lightweight controller that governs each incoming user event to ensure the memory system reflects the user's true, updated belief state. To evaluate how memory systems manage user memory, we introduce a controlled dataset of long-horizon user trajectories with varying persistence of facts based on changing belief states and user-behavioral shifts, and probe queries that test recall while information is valid and abstention after it expires. Our policy learns to effectively manage user memory over its entire lifecycle, balancing recall and abstention to achieve 86% of oracle performance, unlike CL and RAG baselines that primarily default to one of the two behaviors. By providing the first system to manage memory directly in model weights, this work establishes a foundation for long-horizon parametric memory.

## RL Post-Training in Diffusion Transformers Is Activation Steering

DevFeed: [RL Post-Training in Diffusion Transformers Is Activation Steering](<https://devfeed.tech/articles/rl-post-training-in-diffusion-transformers-is-activation-steering-145203.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.26627>)

Author: Tuna Han Salih Meral, Kaan Oktay, Alec Helbling, Atakan Kara, Duen Horng Chau, Pinar Yanardag

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [diffusion-transformers](<https://devfeed.tech/topics/diffusion-transformers.md>), [post-training](<https://devfeed.tech/topics/post-training.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Transformers](<https://devfeed.tech/topics/transformers.md>)

Tags: [activation-steering](<https://devfeed.tech/tags/activation-steering.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [diffusion-transformers](<https://devfeed.tech/tags/diffusion-transformers.md>), [post-training](<https://devfeed.tech/tags/post-training.md>), [rl](<https://devfeed.tech/tags/rl.md>), [steering](<https://devfeed.tech/tags/steering.md>)

### AI overview

The study finds that fixed activation shifts can reproduce most reported gains from RL post-training diffusion transformers, without adding prompt-dependent transformations. The authors add steering vectors to outputs of a frozen base model and fit them to match RL-tuned checkpoints' velocity predictions, without using reward in that fitting objective. On SD3.5-M, the vectors use 0.29M parameters versus 18.8M for the RL-tuned LoRA. Despite being identical across prompts, tokens, and sampling seeds, they recover 90.0-96.3% of reward gains for six image models and 89.8% and 111.9% for two video models. In separate online RL experiments, zero-initialized vectors exceed weight-based RL's reported gains on OCR text rendering and PickScore human preference for SD3.5-M. The abstract's conclusion is that a constant shift can steer the pretrained generator toward capabilities it already computes.

### Source excerpt

arXiv:2611.26627v1 Announce Type: cross Abstract: Reinforcement learning (RL) post-training substantially improves the visual quality, text rendering, and compositional accuracy of diffusion transformers. Because capabilities such as rendering requested text or composing multiple objects are inherently prompt-dependent, a natural hypothesis is that improving them requires learning complex, input-dependent transformations. To isolate how much of these gains an input-independent shift explains, we probe each RL-tuned model with a deliberately simple parameterization: a fixed set of steering vectors added to the block outputs of the frozen base model. On SD3.5-M, these vectors hold 65x fewer parameters than the RL-tuned LoRA (0.29M against 18.8M). We fit them to match the velocity predictions of public RL-tuned checkpoints, with no reward in the objective. Although the vectors are identical for every prompt, token, and sampling seed, they recover 90.0-96.3% of the reward gains of six RL-tuned image models, and 89.8% and 111.9% on two RL-tuned video models. Initialized at zero and trained with online RL, the vectors alone exceed the reward gains of weight-based RL on text rendering (OCR) and human preference (PickScore) on SD3.5-M. A constant shift thus suffices for most of the gain of policy-gradient RL post-training: without new input-dependent transformations, it steers the pretrained generator toward what it can already compute.

## From Geographic Memory to Spatial Generalization in Large Language Models

DevFeed: [From Geographic Memory to Spatial Generalization in Large Language Models](<https://devfeed.tech/articles/from-geographic-memory-to-spatial-generalization-in-large-language-models-145092.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.25703>)

Author: Chen Chu, Zhaonan Wang, Cyrus Shahabi, Hengcai Zhang, Feng Lu

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [geospatial](<https://devfeed.tech/topics/geospatial.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [LLM evaluation / benchmarking](<https://devfeed.tech/topics/llm-evaluation-benchmarking.md>)

Tags: [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [clustering](<https://devfeed.tech/tags/clustering.md>), [continuous](<https://devfeed.tech/tags/continuous.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [geospatial](<https://devfeed.tech/tags/geospatial.md>), [grounding](<https://devfeed.tech/tags/grounding.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [location](<https://devfeed.tech/tags/location.md>), [london](<https://devfeed.tech/tags/london.md>), [spatial](<https://devfeed.tech/tags/spatial.md>)

### AI overview

The study separates geospatial memory--recalling observed location-description pairs--from spatial generalization to locations outside the training observations. It adds continuous location representations from a spatial encoder as tokens in LLMs and evaluates them on newly curated Los Angeles and London benchmarks. Captioning tests observed and unobserved locations, while a separate grounding task measures whether generated pairs help map text to geography. Across multiple LLM families, continuous representations improve captioning at unobserved locations compared with textual coordinates; generated descriptions bring modest but consistent grounding gains. Probing and clustering results are also consistent with spatially organized representations. The evidence is limited to the described benchmarks and assessments.

### Source excerpt

arXiv:2611.25703v1 Announce Type: cross Abstract: Large language models (LLMs) can associate places with geographic descriptions, but recovering an observed association does not establish an ability to generalize across geographic space. We distinguish geospatial memory, the recovery of location--description associations observed during training, from spatial generalization, the prediction of descriptions for locations in unobserved regions. We investigate whether continuous location representations improve this generalization and encourage geographic knowledge to become spatially organized within an LLM. We frame this organization as a latent spatial field: a learned mapping from locations to descriptions connected through spatial dependencies. Building on a continuous spatial encoder, we project location representations into the model's embedding space and supply them as spatial tokens. We evaluate the resulting models on two newly curated geo-text benchmarks for Los Angeles and London using two complementary assessments. Geospatial captioning tests both memory and spatial generalization by generating descriptions at observed locations and at locations beyond the spatial coverage of the training observations. A separate geospatial grounding model tests whether generated location--description pairs provide useful additional supervision for mapping text to geographic space. Across multiple LLM families, continuous location representations improve captioning at unobserved locations relative to textual coordinates, while their generated descriptions yield modest but consistent improvements in downstream grounding. Layer-wise probing and representation clustering provide additional evidence consistent with spatially organized geographic representations. Together, these findings show the value of evaluating spatial generalization separately from memory and of incorporating continuous spatial representations into language models.

## Learning Which Evidence-Seeking Decisions Matter: Calibrated Counterfactual Replay for Language Agents

DevFeed: [Learning Which Evidence-Seeking Decisions Matter: Calibrated Counterfactual Replay for Language Agents](<https://devfeed.tech/articles/learning-which-evidence-seeking-decisions-matter-calibrated-counterfactual-replay-for-language-agents-144902.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2606.05263>)

Author: Renwei Meng, Chengyi Sun, Hongyi Zhang, Xiaoyi Hou

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>), [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [Calibrated uncertainty](<https://devfeed.tech/topics/calibrated-uncertainty.md>)

Tags: [agents](<https://devfeed.tech/tags/agents.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [learning](<https://devfeed.tech/tags/learning.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>)

### AI overview

PCCC-RL estimates which evidence-seeking actions matter by restoring a saved tool state, comparing the chosen action with a feasible alternative, and letting the same frozen policy complete both branches; the difference in final score estimates the action's value. To reduce replay costs, it predicts credit, checks a random subset, and corrects prediction errors using recorded selection probabilities. Alternatives depend only on the prefix, preserving a usable gradient baseline. On HotpotQA and MuSiQue, joint exact match--requiring both a correct answer and complete supporting evidence--was 56.71% and 36.82%, respectively, 4.19 and 3.86 points above a baseline with the same replay access. Gains concentrate in complete evidence chains and accumulate over many updates; single-update effects remain uncertain, while citation coverage and distinctions such as "none" versus "not all" remain open.

### Source excerpt

arXiv:2606.05263v2 Announce Type: replace Abstract: Language agents answer questions by searching, reading, and checking claims. A reward on the final answer says little about which of those decisions earned it, since an agent can reach the right string after redundant searches or with half the evidence missing. We treat the question as an experiment. At a saved decision point, PCCC-RL, for prefix-conditioned counterfactual credit, restores the tool state, runs the action the agent chose alongside a feasible alternative, and lets the same frozen policy finish both branches. The difference in final score then samples what the decision was worth. Replaying everything is too expensive, so the method predicts each action's credit up front, checks a random subset, and uses the recorded selection probabilities to correct the errors of that prediction, spending the budget where a further check can still move the training signal. Alternatives are drawn from the prefix alone, which keeps the comparison a usable gradient baseline. On HotpotQA and MuSiQue, joint exact match, requiring a correct answer together with complete supporting evidence, reaches 56.71% and 36.82%, gains of 4.19 and 3.86 points over a baseline with the same replay access. The gain concentrates in complete evidence chains and accumulates over many updates. Single-update effects stay uncertain, and citation coverage and scope distinctions such as "none" versus "not all" remain open.

## Variational analysis of determinantal varieties

DevFeed: [Variational analysis of determinantal varieties](<https://devfeed.tech/articles/variational-analysis-of-determinantal-varieties-144866.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2511.22613>)

Author: Yan Yang, Bin Gao, Ya-xiang Yuan

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Operations research and optimization](<https://devfeed.tech/topics/operations-research-and-optimization.md>), [Optimization](<https://devfeed.tech/topics/optimization.md>)

Tags: [analysis](<https://devfeed.tech/tags/analysis.md>), [applications](<https://devfeed.tech/tags/applications.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [low-rank](<https://devfeed.tech/tags/low-rank.md>), [math-oc](<https://devfeed.tech/tags/math-oc.md>), [matrix](<https://devfeed.tech/tags/matrix.md>), [np](<https://devfeed.tech/tags/np.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [tangent](<https://devfeed.tech/tags/tangent.md>)

### AI overview

The paper develops explicit first- and second-order tangent-set formulas for low-rank matrices, tensors, symmetric matrices, and positive semidefinite matrices, including an intersection rule under stated assumptions. It uses these tools to derive optimality conditions and a criterion for when a nonsmooth problem and its smooth parameterization have equivalent second-order stationary points; it also proves that checking second-order optimality is NP-hard. For low-rank semidefinite programs, it shows that fourth-order stationary points under the Burer-Monteiro parameterization map to second-order stationary points of the original problem. Separately, it characterizes tangent and normal cones to the graph of the normal cone for matrix varieties and applies these results to low-rank bilevel optimality conditions.

### Source excerpt

arXiv:2511.22613v3 Announce Type: replace-cross Abstract: Determinantal varieties---the sets of bounded-rank matrices or tensors---have attracted growing interest in low-rank optimization. The tangent cone to low-rank sets is widely studied and underpins a range of geometric methods. The second-order geometry, which encodes curvature information, is more intricate. In this work, we develop a unified framework to derive explicit formulas for both first- and second-order tangent sets to various low-rank sets, including low-rank matrices, tensors, symmetric matrices, and positive semidefinite matrices. The framework also accommodates the intersection of a low-rank set and another set satisfying some assumptions, thereby yielding a tangent intersection rule. Through the lens of tangent sets, we establish a necessary and sufficient condition under which a nonsmooth problem and its smooth parameterization share equivalent second-order stationary points. As applications, we exploit tangent sets to characterize optimality conditions for low-rank optimization and prove that verifying second-order optimality is NP-hard. Moreover, for low-rank semidefinite programs, the Burer--Monteiro parameterization maps fourth-order stationary points of the parameterized problem to second-order stationary points of the original problem. In a separate line of analysis, we investigate variational geometry of the graph of the normal cone to matrix varieties, deriving the explicit Bouligand tangent cone, Fr\'echet and Mordukhovich normal cones to the graph. These results are further applied to develop optimality conditions for low-rank bilevel programs.

## Measurable Does Not Imply Transportable: A Prospective Audit of Model Explanations in Multi-Year Field Trials

DevFeed: [Measurable Does Not Imply Transportable: A Prospective Audit of Model Explanations in Multi-Year Field Trials](<https://devfeed.tech/articles/measurable-does-not-imply-transportable-a-prospective-audit-of-model-explanations-in-multi-year-field-trials-145091.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.25692>)

Author: Rayhaneh Shabani Nia, Ali Karkehabadi

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [reproducibility](<https://devfeed.tech/topics/reproducibility.md>), [machine learning overfitting](<https://devfeed.tech/topics/machine-learning-overfitting.md>)

Tags: [2020](<https://devfeed.tech/tags/2020.md>), [2023](<https://devfeed.tech/tags/2023.md>), [alignment](<https://devfeed.tech/tags/alignment.md>), [audit](<https://devfeed.tech/tags/audit.md>), [bootstrap](<https://devfeed.tech/tags/bootstrap.md>), [concepts](<https://devfeed.tech/tags/concepts.md>), [context](<https://devfeed.tech/tags/context.md>), [cosine](<https://devfeed.tech/tags/cosine.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [data](<https://devfeed.tech/tags/data.md>), [environment](<https://devfeed.tech/tags/environment.md>), [failure](<https://devfeed.tech/tags/failure.md>), [feature](<https://devfeed.tech/tags/feature.md>), [feature-importance](<https://devfeed.tech/tags/feature-importance.md>), [field](<https://devfeed.tech/tags/field.md>), [fine](<https://devfeed.tech/tags/fine.md>), [fixed](<https://devfeed.tech/tags/fixed.md>), [future](<https://devfeed.tech/tags/future.md>), [half](<https://devfeed.tech/tags/half.md>), [model](<https://devfeed.tech/tags/model.md>), [transportable](<https://devfeed.tech/tags/transportable.md>)

### AI overview

In multi-year Genomes-to-Fields maize trials, reliable measurement and reproducible explanations within a year did not guarantee that explanations would carry over to future years. Coarse environment-level explanations transported well (median cosine 0.83; locked 2024 value 0.811 within the prespecified interval), while genotype-conditioned grain-yield explanations varied widely across years (median 0.30; range 0.006-0.822). GxE explanations were reproducible within a fixed context (0.77-0.95) but aligned weakly across 2020-2023 transitions (0.01-0.36). The year with the highest split-half reliability had the lowest explanation transport, and non-temporal validation overstated interaction-level transport by 0.22. A second phenotype transported strongly at both resolutions. The abstract concludes that reliability and within-context reproducibility alone do not establish that an explanation supports a reproducible scientific claim.

### Source excerpt

arXiv:2611.25692v1 Announce Type: new Abstract: Model explanations are increasingly used as scientific evidence, yet it remains unclear what they license scientists to conclude beyond the data on which they were constructed. When a fine-grained explanation fails to replicate, the failure may be attributed to noisy measurement of its target, raising a fundamental question: does reliable measurement imply a reproducible explanation? We test this directly. We formalize prospective explanation transportability: the extent to which an explanation learned from past data is recovered in a genuinely future context. This property is distinct from feature importance and measurement reliability. Using multi-year Genomes-to-Fields maize trials, where each new year is a real future context rather than a random resample, we evaluate 16 agronomically interpretable environmental concepts across five forward splits and one prospectively locked test year. Coarse environment-level explanations transport reliably (median cosine 0.83); their locked 2024 value, 0.811, fell within a prespecified 80% bootstrap percentile interval. For grain yield, genotype-conditioned explanations have a historical median cosine of 0.30 and range from 0.006 to 0.822 across years. Reliability does not order these outcomes: the year with the highest split-half reliability had the lowest explanation transport. Within-context resampling showed that GxE explanations were reproducible within a fixed context (0.77-0.95) while exhibiting lower prospective alignment across the 2020-2023 transitions (0.01-0.36). A second phenotype transports strongly at both resolutions, and non-temporal validation inflates apparent interaction-level transport by +0.22. These results show that target reliability and within-context explanation reproducibility do not by themselves establish when a model explanation can support a reproducible scientific claim.

## Survey-aware Machine Learning: A Guideline for Valid Population Health Inference based on Scoping Review

DevFeed: [Survey-aware Machine Learning: A Guideline for Valid Population Health Inference based on Scoping Review](<https://devfeed.tech/articles/survey-aware-machine-learning-a-guideline-for-valid-population-health-inference-based-on-scoping-review-144893.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2605.08963>)

Author: YongKyung Oh, Henry W. Zheng, Jeffrey Feng, Alex A. T. Bui

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Model Development](<https://devfeed.tech/topics/model-development.md>), [Hyperparameter optimization](<https://devfeed.tech/topics/hyperparameter-optimization.md>)

Tags: [cross-validation](<https://devfeed.tech/tags/cross-validation.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [deployment](<https://devfeed.tech/tags/deployment.md>), [design](<https://devfeed.tech/tags/design.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [fairness](<https://devfeed.tech/tags/fairness.md>), [guidance](<https://devfeed.tech/tags/guidance.md>), [health](<https://devfeed.tech/tags/health.md>), [inference](<https://devfeed.tech/tags/inference.md>), [lifecycle](<https://devfeed.tech/tags/lifecycle.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [metadata](<https://devfeed.tech/tags/metadata.md>), [ml](<https://devfeed.tech/tags/ml.md>), [model-training](<https://devfeed.tech/tags/model-training.md>), [models](<https://devfeed.tech/tags/models.md>), [nutrition](<https://devfeed.tech/tags/nutrition.md>), [objectives](<https://devfeed.tech/tags/objectives.md>), [performance](<https://devfeed.tech/tags/performance.md>), [performance-evaluation](<https://devfeed.tech/tags/performance-evaluation.md>), [review](<https://devfeed.tech/tags/review.md>), [stat-ml](<https://devfeed.tech/tags/stat-ml.md>), [survey](<https://devfeed.tech/tags/survey.md>)

### AI overview

Standard ML evaluation on complex health surveys can produce biased estimates, understate uncertainty, and miss population-level disparities when it ignores sampling units, stratification, and weights. The paper proposes Survey-aware Machine Learning (SaML), a nine-step guideline for incorporating survey design metadata across the ML lifecycle. Its scoping review of 16 methodological papers summarizes weighted training, design-based cross-validation, and survey-adjusted performance evaluation, while identifying gaps in hyperparameter tuning and deployment. The guideline offers task-specific direction on which steps are needed for different analytical objectives.

### Source excerpt

arXiv:2605.08963v2 Announce Type: replace-cross Abstract: Machine Learning (ML) models trained on complex health surveys such as the National Health and Nutrition Examination Survey (NHANES) often ignore primary sampling units, stratification variables, and sampling weights. This practice violates the independence assumptions of standard evaluation methods. As a result, estimates become biased, uncertainty is underestimated, and fairness assessments fail to reflect population-level disparities. We propose Survey-aware Machine Learning (SaML), a nine-step guideline that incorporates survey design metadata across the ML lifecycle. Through a scoping review of 16 methodological papers, we summarize existing work on weighted model training, design-based cross-validation, and survey-adjusted performance evaluation. We also identify gaps in hyperparameter tuning and deployment. We provide task-specific guidance that clarifies which steps are required for different analytical objectives. SaML provides a checklist for valid population inference from survey data.

## Attention Head Specialization: Phase Transitions in Sample Complexity

DevFeed: [Attention Head Specialization: Phase Transitions in Sample Complexity](<https://devfeed.tech/articles/attention-head-specialization-phase-transitions-in-sample-complexity-145153.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.26228>)

Author: L\\'eo Catteau, O. Duranthon, Lenka Zdeborov\\'a

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Transformer](<https://devfeed.tech/topics/transformer.md>), [Neural Network](<https://devfeed.tech/topics/neural-network.md>)

Tags: [attention](<https://devfeed.tech/tags/attention.md>), [bert](<https://devfeed.tech/tags/bert.md>), [bias](<https://devfeed.tech/tags/bias.md>), [cascade](<https://devfeed.tech/tags/cascade.md>), [changes](<https://devfeed.tech/tags/changes.md>), [complexity](<https://devfeed.tech/tags/complexity.md>), [cond-mat-dis-nn](<https://devfeed.tech/tags/cond-mat-dis-nn.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [data](<https://devfeed.tech/tags/data.md>), [data-structure](<https://devfeed.tech/tags/data-structure.md>), [dataset](<https://devfeed.tech/tags/dataset.md>), [emergence](<https://devfeed.tech/tags/emergence.md>), [error](<https://devfeed.tech/tags/error.md>), [features](<https://devfeed.tech/tags/features.md>), [function](<https://devfeed.tech/tags/function.md>), [learning](<https://devfeed.tech/tags/learning.md>), [model](<https://devfeed.tech/tags/model.md>), [multi-head](<https://devfeed.tech/tags/multi-head.md>), [online](<https://devfeed.tech/tags/online.md>), [online-learning](<https://devfeed.tech/tags/online-learning.md>), [sample](<https://devfeed.tech/tags/sample.md>), [specialization](<https://devfeed.tech/tags/specialization.md>), [stat-ml](<https://devfeed.tech/tags/stat-ml.md>)

### AI overview

The excerpt reports that BERT heads specialized to syntactic relations emerge as training dataset size increases. In a solvable high-dimensional multi-head attention model, specialization is a phase transition in sample complexity, marked by non-analytic changes in test error and other observables. Depending on data structure and regularization, heads may differentiate without conveying useful information, or specialize through successive transitions that recover progressively weaker latent features. The authors also relate these statistical transitions to stagewise specialization in online learning, separating statistical effects from training dynamics.

### Source excerpt

arXiv:2611.26228v1 Announce Type: new Abstract: Attention heads specialize into distinct functions during training, often emerging in stages together with sharp changes in model behavior. While this phenomenon has mostly been studied as a function of training time, we ask how head specialization depends on the amount of available training data. Empirically, we find that heads specialized to syntactic relations in BERT emerge as the dataset size increases. We then study this phenomenon in a solvable high-dimensional model of multi-head attention and establish that, in this model, head specialization is a genuine phase transition in sample complexity, corresponding to a non-analyticity in the test error and other observables. Depending on data structure and regularization, specialization can follow uninformative head differentiation or occur through a cascade of phase transitions that progressively recover weaker latent features, providing a statistical form of simplicity bias. Finally, we relate these statistical transitions to the stagewise emergence of specialized heads during online learning, disentangling statistical from dynamical origins of head specialization.

## Approximate Adjoint Models for Memory and Compute-Efficient Online Learning of Hybrid Models

DevFeed: [Approximate Adjoint Models for Memory and Compute-Efficient Online Learning of Hybrid Models](<https://devfeed.tech/articles/approximate-adjoint-models-for-memory-and-compute-efficient-online-learning-of-hybrid-models-145140.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.26095>)

Author: Emilio Gonz\\'alez-Zamora, Pierre Tandeo, Laurent Debreu, Said Ouala

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Model Development](<https://devfeed.tech/topics/model-development.md>)

Tags: [approximation](<https://devfeed.tech/tags/approximation.md>), [backpropagation](<https://devfeed.tech/tags/backpropagation.md>), [bptt](<https://devfeed.tech/tags/bptt.md>), [closure](<https://devfeed.tech/tags/closure.md>), [compute](<https://devfeed.tech/tags/compute.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [hybrid](<https://devfeed.tech/tags/hybrid.md>), [learning](<https://devfeed.tech/tags/learning.md>), [math-oc](<https://devfeed.tech/tags/math-oc.md>), [memory](<https://devfeed.tech/tags/memory.md>), [model-training](<https://devfeed.tech/tags/model-training.md>), [ocean](<https://devfeed.tech/tags/ocean.md>), [online](<https://devfeed.tech/tags/online.md>), [online-learning](<https://devfeed.tech/tags/online-learning.md>), [physics-ao-ph](<https://devfeed.tech/tags/physics-ao-ph.md>)

### AI overview

The proposed methods reduce the memory and compute costs of online hybrid-model training while preserving adjoint recurrence across the full trajectory. They compute exact pullbacks only at sparse anchor states; between anchors, Zero-order propagates sensitivities through the identity, while Pullback Reuse uses a frozen pullback. Reusing the pullback also lets the parameter-gradient sum factorize to one evaluation per chunk instead of one per time step. Across an incomplete Lorenz-63 model, a two-dimensional quasi-geostrophic turbulence closure, and bias correction in a multilayer quasi-geostrophic ocean model, peak memory fell by up to an order of magnitude and runtime by up to fourfold. Pullback Reuse stayed closest to the exact gradient direction, and both methods supported training at comparable accuracy. These are abstract-level reported results; no further experimental details are provided here.

### Source excerpt

arXiv:2611.26095v1 Announce Type: new Abstract: Differentiable programming enables online calibration of numerical models by minimizing errors accumulated along simulated trajectories. This includes estimating physical parameters and training neural sub-models in hybrid configurations. Gradients are typically computed using adjoint methods, or backpropagation through time (BPTT), whose computational and memory costs grow with the length of the training window. This limits the duration and resolution of hybrid-model training, a major bottleneck for realistic geophysical applications. Approaches like truncated BPTT reduces memory use by discarding sensitivities across truncation boundaries, introducing gradient bias and neglecting long-range dependencies. We propose a class of gradient approximation methods that preserve the adjoint recurrence over the full trajectory while approximating only the pullback operators within it. Exact pullbacks are computed by automatic differentiation at sparse anchor states only. Between anchors, sensitivities propagate either through the identity (Zero-order) or through a reused, frozen pullback (Pullback Reuse). This approximation makes the parameter pullback constant within each chunk, allowing the gradient sum to factorize so that a single evaluation per chunk replaces one per time step. We evaluate both on three systems of increasing complexity, an incomplete Lorenz-63 model, a large-eddy closure for two-dimensional quasi-geostrophic turbulence, and bias correction in a multilayer quasi-geostrophic ocean model. Peak memory falls by up to an order of magnitude and runtime up to fourfold, with Pullback Reuse staying closest to the exact gradient direction. In all three cases, approximate gradients support hybrid-model training at comparable accuracy, showing that effective online learning does not require exact gradients.

## Achieving Near-Zero-Overhead Multi-Model Hierarchical Classification in Real-Time Detection Pipelines

DevFeed: [Achieving Near-Zero-Overhead Multi-Model Hierarchical Classification in Real-Time Detection Pipelines](<https://devfeed.tech/articles/achieving-near-zero-overhead-multi-model-hierarchical-classification-in-real-time-detection-pipelines-144919.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2608.11770>)

Author: Vaishnav Raju

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Distributed Computing](<https://devfeed.tech/sources/arxiv-distributed-computing.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [quantization](<https://devfeed.tech/topics/quantization.md>), [Jetson Orin](<https://devfeed.tech/topics/jetson-orin.md>), [AI Integration Strategies](<https://devfeed.tech/topics/ai-integration-strategies.md>), [real-time](<https://devfeed.tech/topics/real-time.md>)

Tags: [accelerators](<https://devfeed.tech/tags/accelerators.md>), [analysis](<https://devfeed.tech/tags/analysis.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [autonomous](<https://devfeed.tech/tags/autonomous.md>), [autonomous-vehicles](<https://devfeed.tech/tags/autonomous-vehicles.md>), [bottleneck](<https://devfeed.tech/tags/bottleneck.md>), [classification](<https://devfeed.tech/tags/classification.md>), [classifier](<https://devfeed.tech/tags/classifier.md>), [classifiers](<https://devfeed.tech/tags/classifiers.md>), [compilation](<https://devfeed.tech/tags/compilation.md>), [concurrent](<https://devfeed.tech/tags/concurrent.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-dc](<https://devfeed.tech/tags/cs-dc.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [custom](<https://devfeed.tech/tags/custom.md>), [deployment](<https://devfeed.tech/tags/deployment.md>), [detection](<https://devfeed.tech/tags/detection.md>), [drone](<https://devfeed.tech/tags/drone.md>), [edge](<https://devfeed.tech/tags/edge.md>), [overhead](<https://devfeed.tech/tags/overhead.md>), [pipelines](<https://devfeed.tech/tags/pipelines.md>), [real-time](<https://devfeed.tech/tags/real-time.md>)

### AI overview

The paper reports that a GPU detector paired with DLA classification on a Jetson Orin NX ran at 12.5 FPS at 1080p, compared with 13.3 FPS for the detector alone. Its five-step deployment method covers adapting classification architectures, repairing TensorRT implicit quantization ranges for pipeline validation (raising reported accuracy from 75% to 94%), quantization-aware training, ONNX graph surgery, and concurrent GPU/DLA inference. The authors also describe nine engineering constraints and report dual-DLA scaling without additional cost. Evidence here is limited to the abstract.

### Source excerpt

arXiv:2608.11770v2 Announce Type: replace-cross Abstract: Edge-deployed vision systems in target recognition, surveillance, autonomous vehicles, and drone domains require hierarchical inference pipelines where a detection model identifies objects of interest and downstream classifiers provide fine-grained attribute analysis. Running all models on the GPU creates a serial bottleneck that limits real-time throughput as pipeline stages grow. Modern edge SoCs pair GPUs with dedicated neural accelerators (NPUs, DLAs) capable of concurrent execution, yet deploying custom models on these accelerators remains impractical due to strict operator constraints, quantization incompatibilities, and an undocumented end-to-end pipeline. We target NVIDIA Jetson DLA cores as the representative platform. We present a five-step methodology for zero GPU fallback DLA INT8 deployment of classification backbones, comprising architecture adaptation, manual dynamic range workaround to rescue TensorRT's implicit quantization (recovering 94.0% accuracy from implicit quantization's 75%) for rapid pipeline validation before explicit quantization, quantization-aware training, ONNX graph surgery for DLA compilation, and a concurrent GPU-detection/DLA-classification inference pipeline. We document nine engineering constraints with root-cause analysis and generalizable solutions. Validation on a dual-head person attribute classifier running on DLA alongside a GPU object detector on a Jetson Orin NX demonstrates near-zero pipeline overhead (12.5 vs. 13.3~FPS detector-only at 1080p), with dual-DLA scaling at no additional cost. The methodology is backbone-agnostic and generalizes to any detection-classification edge pipeline.

## Stop Removing Stopwords: How an Inherited Preprocessing Default Distorts Legal Text-as-Data

DevFeed: [Stop Removing Stopwords: How an Inherited Preprocessing Default Distorts Legal Text-as-Data](<https://devfeed.tech/articles/stop-removing-stopwords-how-an-inherited-preprocessing-default-distorts-legal-text-as-data-144926.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2609.19153>)

Author: Gregory M. Dickinson

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Data science and engineering](<https://devfeed.tech/topics/data-science-and-engineering.md>)

Tags: [ablation](<https://devfeed.tech/tags/ablation.md>), [classification](<https://devfeed.tech/tags/classification.md>), [classifiers](<https://devfeed.tech/tags/classifiers.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-cy](<https://devfeed.tech/tags/cs-cy.md>), [cs-ir](<https://devfeed.tech/tags/cs-ir.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [database](<https://devfeed.tech/tags/database.md>), [f1](<https://devfeed.tech/tags/f1.md>), [information-retrieval](<https://devfeed.tech/tags/information-retrieval.md>), [law](<https://devfeed.tech/tags/law.md>), [legal](<https://devfeed.tech/tags/legal.md>), [stop](<https://devfeed.tech/tags/stop.md>), [text](<https://devfeed.tech/tags/text.md>), [tf-idf](<https://devfeed.tech/tags/tf-idf.md>)

### AI overview

An exhaustive single-word ablation found no detectable benefit from stopword removal in two Supreme Court opinion classification tasks: ideological direction (7,668 opinions; baseline F1 about 0.68) and constitutional versus non-constitutional law type (7,001; about 0.92). The study tested roughly 18,500 words per task. Generic stoplists fell below the no-removal baseline in all twelve held-out tests; task-specific stoplists changed held-out F1 by only +0.0023 for ideology (95% CI −0.0124 to +0.0170) and +0.0001 for law type (−0.0082 to +0.0085). The authors argue that, in interpretable legal text-as-data pipelines, inherited preprocessing choices should be evaluated against the downstream objective because reshaping available features may alter the signal researchers aim to recover.

### Source excerpt

arXiv:2609.19153v3 Announce Type: replace-cross Abstract: Empirical legal scholarship increasingly treats judicial text as data, and much of it still runs on sparse, interpretable pipelines -- TF-IDF features and linear classifiers -- because the textual feature is often the object of study rather than a means to prediction. Yet these pipelines inherit preprocessing defaults from mid-century information retrieval that were never validated against classification accuracy. The most entrenched of these is stopword removal. This study introduces an exhaustive single word ablation that measures the effect of a preprocessing step directly against the downstream objective, and applies it to stopword removal as the hardest case to dislodge. Matching Supreme Court Database labels to Caselaw Access Project opinion texts, the study examines two binary tasks that bracket F1 headroom, ideological direction (no-removal baseline F1 ~0.68) and constitutional versus non-constitutional law type (~0.92), across 7,668 and 7,001 opinions. For each task, the ablation removes each of about 18,500 candidate words in turn, and a task-specific stoplist is built from the measurements. Common generic stoplists fall below the no-removal baseline in all twelve held-out tests. The task-specific stoplists move held-out F1 by +0.0023 (95% CI [-0.0124, +0.0170]) on ideology and by +0.0001 ([-0.0082, +0.0085]) on law type. Neither task shows a detectable benefit from removal, and supplemental analysis finds that word-level statistics predict removal effects poorly, because true removal effects differ by less than the measurement can register, leaving little to predict. The method generalizes to any inherited preprocessing default; the result is a caution specific to interpretable legal text-as-data, where a step that reshapes which features a model sees can distort the doctrinal and ideological signal the research is meant to recover. The burden of proof sits with removal.

## Optimal algorithmic complexity of inference in quantum kernel methods

DevFeed: [Optimal algorithmic complexity of inference in quantum kernel methods](<https://devfeed.tech/articles/optimal-algorithmic-complexity-of-inference-in-quantum-kernel-methods-144886.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2604.15214>)

Author: Elies Gil-Fuster, Seongwook Shin, Sofiene Jerbi, Jens Eisert, Maximilian J. Kramer

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Inference](<https://devfeed.tech/topics/inference.md>), [Algorithms](<https://devfeed.tech/topics/algorithms.md>), [Combinatorial optimization](<https://devfeed.tech/topics/combinatorial-optimization.md>)

Tags: [algorithm](<https://devfeed.tech/tags/algorithm.md>), [algorithms](<https://devfeed.tech/tags/algorithms.md>), [amplitude](<https://devfeed.tech/tags/amplitude.md>), [bottleneck](<https://devfeed.tech/tags/bottleneck.md>), [complexity](<https://devfeed.tech/tags/complexity.md>), [cost](<https://devfeed.tech/tags/cost.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [encoding](<https://devfeed.tech/tags/encoding.md>), [gate](<https://devfeed.tech/tags/gate.md>), [inference](<https://devfeed.tech/tags/inference.md>), [kernel](<https://devfeed.tech/tags/kernel.md>), [lower-bound](<https://devfeed.tech/tags/lower-bound.md>), [quant-ph](<https://devfeed.tech/tags/quant-ph.md>), [quantum](<https://devfeed.tech/tags/quantum.md>), [quantum-advantage](<https://devfeed.tech/tags/quantum-advantage.md>), [query](<https://devfeed.tech/tags/query.md>), [sampling](<https://devfeed.tech/tags/sampling.md>), [supervised-learning](<https://devfeed.tech/tags/supervised-learning.md>)

### AI overview

Quantum kernel inference estimates a weighted sum of kernel values, and independently sampling each term has query complexity O(N‖α‖₂²/ε²). The abstract identifies two choices--sampling or quantum amplitude estimation for kernel values, and term-by-term estimation or encoding the full sum as one observable. Combining the single-observable encoding with amplitude estimation achieves O(‖α‖₁/ε) queries, with a matching lower bound up to logarithmic factors. This removes N from the query count and improves dependence on ‖α‖₁ and ε. The paper also analyzes gate costs: the query-optimal method is not always best in gate complexity, so the practical strategy depends on hardware. All described algorithms use amplitude estimation as a subroutine.

### Source excerpt

arXiv:2604.15214v3 Announce Type: replace-cross Abstract: Quantum kernel methods are among the leading candidates for achieving quantum advantage in supervised learning. A key bottleneck is the cost of inference: evaluating a trained model on new data requires estimating a weighted sum $\sum_{i=1}^N \alpha_i k(x,x_i)$ of $N$ kernel values to additive precision $\varepsilon$, where $\alpha$ is the vector of trained coefficients. The standard approach estimates each term independently via sampling, yielding a query complexity of $O(N\lVert\alpha\rVert_2^2/\varepsilon^2)$. In this work, we identify two independent axes for improvement: (1) How individual kernel values are estimated (sampling versus quantum amplitude estimation), and (2) how the sum is approximated (term-by-term versus via a single observable), and systematically analyze all combinations thereof. The query-optimal combination, encoding the full inference sum as the expectation value of a single observable and applying quantum amplitude estimation, achieves a query complexity of $O(\lVert\alpha\rVert_1/\varepsilon)$, removing the dependence on $N$ from the query count and yielding a quadratic improvement in both $\lVert\alpha\rVert_1$ and $\varepsilon$. We prove a matching lower bound of $\Omega(\lVert\alpha\rVert_1/\varepsilon)$, establishing query-optimality of our approach up to logarithmic factors. Beyond query complexity, we also analyze how these improvements translate into gate costs and show that the query-optimal strategy is not always optimal in practice from the perspective of gate complexity. Our results provide both a query-optimal algorithm and a practically optimal choice of strategy depending on hardware capabilities, along with a complete landscape of intermediate methods to guide practitioners. All algorithms require only amplitude estimation as a subroutine and are thus natural candidates for early-fault-tolerant implementations.

## Tetra: Serving Leech-Lattice Quantized LLMs at 2.7 Bits per Parameter

DevFeed: [Tetra: Serving Leech-Lattice Quantized LLMs at 2.7 Bits per Parameter](<https://devfeed.tech/articles/tetra-serving-leech-lattice-quantized-llms-at-2-7-bits-per-parameter-144940.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2609.35465>)

Author: Pier-Jean Malandrino

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [quantization](<https://devfeed.tech/topics/quantization.md>), [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>)

Tags: [awq](<https://devfeed.tech/tags/awq.md>), [block](<https://devfeed.tech/tags/block.md>), [code](<https://devfeed.tech/tags/code.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [embedding](<https://devfeed.tech/tags/embedding.md>), [engine](<https://devfeed.tech/tags/engine.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [figure](<https://devfeed.tech/tags/figure.md>), [file](<https://devfeed.tech/tags/file.md>), [files](<https://devfeed.tech/tags/files.md>), [fp16](<https://devfeed.tech/tags/fp16.md>), [full](<https://devfeed.tech/tags/full.md>), [generate](<https://devfeed.tech/tags/generate.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [gpu-memory](<https://devfeed.tech/tags/gpu-memory.md>), [kernel](<https://devfeed.tech/tags/kernel.md>), [lattice](<https://devfeed.tech/tags/lattice.md>), [llms](<https://devfeed.tech/tags/llms.md>), [lookup](<https://devfeed.tech/tags/lookup.md>), [serving](<https://devfeed.tech/tags/serving.md>), [tetra](<https://devfeed.tech/tags/tetra.md>)

### AI overview

Tetra makes Leech-lattice quantization practical for GPU inference by encoding each 24-weight block in 48 bits and decoding it during matrix-vector multiplication with six table loads and two small lookups. Its shared 16 KiB table avoids the enormous full codebook, and the kernel reads 2.148 bits per weight, compared with 4.804 bits per weight for the authors' earlier load-time expansion approach. Full Qwen3-4B, 8B and 14B models occupy 2.70-2.73 bits per parameter, using row-specific scales, 4-bit storage for the matrices that lose most quality, and 4-bit embedding tables. On MMLU they trail 4-bit AWQ models (5.3-6.0 bits per parameter) by 2.46-4.76 points; on GSM8K they trail FP16 by 3.26-9.63 points. Reported generation rates are 57.2-113.8 tokens per second. At 4B, Tetra scores 23.6 points above llama.cpp IQ2_XXS. All reported measurements used one NVIDIA L40S GPU.

### Source excerpt

arXiv:2609.35465v2 Announce Type: replace Abstract: Leech-lattice quantization gives good quality at two bits per weight, but its codebooks hold more than 10^14 points, too many for a lookup table. Our earlier kernel expanded the codes at load time and read 4.804 bits per weight from GPU memory for 2 bits of code. We present Tetra, a new codebook on the same lattice. A 24-weight block still takes 48 bits, most of which index a 64-state trellis of the Golay code and one shared 16 KiB table. The kernel decodes a block with six table loads and two small lookups inside the matrix-vector product, and reads 2.148 bits per weight. For full models, we retrain one scale per matrix row, store the matrices that lose the most as 4-bit integers, and pay for them with 4-bit embedding tables. Our Qwen3-4B, 8B and 14B files hold 2.73, 2.70 and 2.73 bits per parameter over the whole model. They score 63.37, 69.58 and 75.66 on the full MMLU test set, 4.76, 4.21 and 2.46 points below 4-bit AWQ at 5.3 to 6.0 bits per parameter. They generate 113.8, 95.0 and 57.2 tokens per second in our engine. On GSM8K, through the served kernel, they lose 9.63, 4.62 and 3.26 points to FP16. At 4B our file scores 23.6 points above llama.cpp's IQ2_XXS (2.48 bits per parameter). Every number we measured for a table or figure comes from one NVIDIA L40S GPU. We preregistered the main experiments.

## Grid Cells as Spectral Successor Features: A Reusable Basis for Continual Reinforcement Learning

DevFeed: [Grid Cells as Spectral Successor Features: A Reusable Basis for Continual Reinforcement Learning](<https://devfeed.tech/articles/grid-cells-as-spectral-successor-features-a-reusable-basis-for-continual-reinforcement-learning-145188.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.26504>)

Author: \\'Elodie C\\^ot\\'e-Gauthier, Doina Precup, Isabeau Pr\\'emont-Schwarz

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>)

Tags: [adapting](<https://devfeed.tech/tags/adapting.md>), [basis](<https://devfeed.tech/tags/basis.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [eigenvectors](<https://devfeed.tech/tags/eigenvectors.md>), [features](<https://devfeed.tech/tags/features.md>), [grid](<https://devfeed.tech/tags/grid.md>), [linear-regression](<https://devfeed.tech/tags/linear-regression.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [policy](<https://devfeed.tech/tags/policy.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [reward](<https://devfeed.tech/tags/reward.md>)

### AI overview

The paper builds a reusable basis for continual reinforcement learning from eigenvectors of a fixed reference policy's transition matrix. In this basis, each successor feature is the corresponding base feature scaled by 1/(1−γλᵢ), so returns use the same features as immediate rewards; task adaptation becomes linear regression on reward, without a successor-feature network or replay buffer. The agent chooses actions greedily using the reference policy's action values, described as one step of policy improvement that suffices for a single goal. In a two-goal gridworld using a uniform-random-walk reference policy, the approach maintained near-maximal return through goal changes, while SimpleSF and DQN lost most of their return at each change. These results are limited to the described setting.

### Source excerpt

arXiv:2611.26504v1 Announce Type: new Abstract: Continual reinforcement learning requires agents to adapt when the task changes while reusing what they know about the environment. Successor features separate dynamics from task-specific reward, but their effectiveness depends on the choice of base features. Taking the identification of entorhinal grid cells with eigenvectors of the successor representation as a design principle, we build this basis from eigenvectors of the transition matrix of a fixed reference policy. With such a basis, the successor features at a state-action pair are a linear combination of the base features at that same pair, which is not true in general, and the coefficients are known in closed form: each successor feature is its base feature multiplied by $\theta_i = 1/(1-\gamma\lambda_i)$, where $\lambda_i$ is its eigenvalue. The long-term return is thus predicted from the same features as the immediate reward, no successor-feature network or replay buffer is needed, and adapting to a new task reduces to a linear regression on the reward. The agent acts greedily on the action values of the reference policy, which is one step of policy improvement over it and suffices to reach a single goal. In a two-goal gridworld, with the uniform random walk as reference policy, whose eigenvectors are the low-frequency Laplacian eigenvectors, agents built this way maintain near-maximal return across every change of goal, whereas SimpleSF and DQN lose most of their return at each change.

## Depth Pro 2: Sharp and Temporally Consistent Metric Monocular Depth

DevFeed: [Depth Pro 2: Sharp and Temporally Consistent Metric Monocular Depth](<https://devfeed.tech/articles/depth-pro-2-sharp-and-temporally-consistent-metric-monocular-depth-145199.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.26603>)

Author: Peiyun Hu, Marko Mihajlovic, Lyndon Duong, Lahav Lipson, Wei Dong, Ama\\"el Delaunoy, Stephan R. Richter, Vladlen Koltun

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Image](<https://devfeed.tech/topics/image.md>), [multimodal-ai](<https://devfeed.tech/topics/multimodal-ai.md>), [3D](<https://devfeed.tech/topics/3d.md>)

Tags: [code](<https://devfeed.tech/tags/code.md>), [consistency](<https://devfeed.tech/tags/consistency.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [depth-estimation](<https://devfeed.tech/tags/depth-estimation.md>), [frontier](<https://devfeed.tech/tags/frontier.md>), [images](<https://devfeed.tech/tags/images.md>), [maps](<https://devfeed.tech/tags/maps.md>), [metric](<https://devfeed.tech/tags/metric.md>), [models](<https://devfeed.tech/tags/models.md>), [native](<https://devfeed.tech/tags/native.md>), [network](<https://devfeed.tech/tags/network.md>), [runtime](<https://devfeed.tech/tags/runtime.md>), [sharp](<https://devfeed.tech/tags/sharp.md>), [temporal](<https://devfeed.tech/tags/temporal.md>), [trade](<https://devfeed.tech/tags/trade.md>), [train](<https://devfeed.tech/tags/train.md>), [video](<https://devfeed.tech/tags/video.md>)

### AI overview

Depth Pro 2 is a scalable network for estimating dense metric depth from images and video at native resolution. The authors identify a common tradeoff in prior monocular methods: multi-frame approaches tend to favor temporal consistency, while single-frame approaches favor sharp boundaries. Their model processes both images and video and is reported to deliver metric accuracy, sharpness, and temporal consistency together, outperforming prior work on those dimensions. The released model family spans a Pareto frontier across metric accuracy, sharpness, temporal consistency, and runtime; the excerpt does not provide quantitative comparisons or details on runtime.

### Source excerpt

arXiv:2611.26603v1 Announce Type: cross Abstract: We present Depth Pro 2, an approach to estimating dense metric depth for images and video at full resolution. By analyzing prior methods for monocular depth estimation, we find that they commonly trade off temporal consistency and boundary sharpness: multi-frame methods favor consistency, single-frame methods favor sharpness. To overcome this trade-off, we train a single, scalable network that processes both images and video and predicts depth maps at native resolution. Depth Pro 2 produces depth maps that are metrically accurate, sharp, and temporally consistent all at once, outperforming prior work in all of these dimensions. We release a family of models that define a new Pareto frontier in terms of metric accuracy, sharpness, temporal consistency, and runtime. We release code and weights at https://apple-aiml-research.github.io/ml-depth-pro-2/

## Reference-Free Humanoid Skateboarding via Termination-Shaped Reinforcement Learning

DevFeed: [Reference-Free Humanoid Skateboarding via Termination-Shaped Reinforcement Learning](<https://devfeed.tech/articles/reference-free-humanoid-skateboarding-via-termination-shaped-reinforcement-learning-145131.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.26054>)

Author: Ganeshvar Soenessardien, Michael Przystupa, Kevin Sebastian Luck

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Humanoid Robots](<https://devfeed.tech/topics/humanoid-robots.md>), [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>), [Robot Manipulation](<https://devfeed.tech/topics/robot-manipulation.md>)

Tags: [board](<https://devfeed.tech/tags/board.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-ro](<https://devfeed.tech/tags/cs-ro.md>), [deep-reinforcement-learning](<https://devfeed.tech/tags/deep-reinforcement-learning.md>), [humanoid](<https://devfeed.tech/tags/humanoid.md>), [interaction](<https://devfeed.tech/tags/interaction.md>), [model](<https://devfeed.tech/tags/model.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>)

### AI overview

The paper presents model-free deep reinforcement learning for teaching the K1 humanoid to skateboard without demonstrations or analytic steering models requiring system identification. It combines task rewards with termination constraints that block behaviors including losing the planted board foot, putting both feet on the board, excessive twisting, separating from the board, and unstable board motion. As these behaviors become less prevalent during training, the constraints produce an implicit progression from stance control toward sustained robot-board locomotion.

### Source excerpt

arXiv:2611.26054v1 Announce Type: cross Abstract: Humanoid locomotion research has advanced rapidly on walking and running, but \emph {object-mediated} locomotion, in which a robot must coordinate its body with a passive external object, remains difficult. One challenging task is skateboarding, which requires simultaneous interaction with the ground and a passive but movable platform to achieve forward motion. Current methods for tackling skateboarding with humanoid robots either require human demonstrations or utilise analytic steering models that require system identification. In this paper, we introduce a deep reinforcement learning framework that does not rely on prior data such as demonstrations and enables robots such as the K1 humanoid to learn to skateboard using model-free learning. The method combines task rewards with exploit-blocking terminations. We introduce a number of constraints restricting behaviours such as losing the planted board foot, standing with both feet on the board, excessive twisting, separation from the board, and unstable board motion. Their changing prevalence during training produces an implicit progression from stance control towards sustained robot--board locomotion.

## D\\'ej\\\`a Cue: Localizing States in Object Histories via Vocabulary-Relative Coordinates

DevFeed: [D\\'ej\\\`a Cue: Localizing States in Object Histories via Vocabulary-Relative Coordinates](<https://devfeed.tech/articles/d-ej-a-cue-localizing-states-in-object-histories-via-vocabulary-relative-coordinates-144916.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2608.02044>)

Author: Haofan Cao, Zhichao You, Yunkai Yang, Liang Guo, Jie Wang, Chongshou Li

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Embeddings](<https://devfeed.tech/topics/embeddings.md>), [Image](<https://devfeed.tech/topics/image.md>)

Tags: [absolute](<https://devfeed.tech/tags/absolute.md>), [alternatives](<https://devfeed.tech/tags/alternatives.md>), [computer-vision](<https://devfeed.tech/tags/computer-vision.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-mm](<https://devfeed.tech/tags/cs-mm.md>), [embeddings](<https://devfeed.tech/tags/embeddings.md>), [encoder](<https://devfeed.tech/tags/encoder.md>), [localization](<https://devfeed.tech/tags/localization.md>), [metrics](<https://devfeed.tech/tags/metrics.md>), [object](<https://devfeed.tech/tags/object.md>), [ranking](<https://devfeed.tech/tags/ranking.md>), [retrieval](<https://devfeed.tech/tags/retrieval.md>), [search](<https://devfeed.tech/tags/search.md>), [semantic](<https://devfeed.tech/tags/semantic.md>), [semantics](<https://devfeed.tech/tags/semantics.md>), [temporal](<https://devfeed.tech/tags/temporal.md>), [text](<https://devfeed.tech/tags/text.md>), [vision](<https://devfeed.tech/tags/vision.md>)

### AI overview

Déjà Cue localizes described states in tracked object histories by comparing each description against the alternatives in a supplied, state-balanced vocabulary. The training-free method centers description embeddings on the vocabulary mean, then uses frozen vision-language encoders to calibrate frame scores and retrieve intervals of multiple durations within contiguous visible runs. The authors introduce MS-VOST, annotating 620 object histories with 2,465 states. With the encoder and temporal search fixed, it outperforms absolute scoring on all three localization metrics across eight encoder configurations, improving Top-1 temporal IoU by 0.42-1.14 percentage points. Reported gains persist across four text constructions; candidate-ranking analysis indicates relevant intervals are ranked higher within the same search space.

### Source excerpt

arXiv:2608.02044v2 Announce Type: replace-cross Abstract: Object tracking preserves identity through visual changes, but cannot tell when an object is whole, cut, or peeled. We study state-moment retrieval: locating when a tracked object exhibits each state in a supplied set of descriptions. However, absolute image--text similarity can favor shared object semantics, obscuring the distinctions needed to localize individual states. Our insight is that the alternatives themselves provide a reference for measuring state evidence. We introduce D\'ej\`a Cue, a training-free framework that centers description embeddings on a state-balanced vocabulary mean, producing query directions that compare each state against its alternatives. Using frozen vision--language encoders, D\'ej\`a Cue calibrates frame scores and retrieves intervals at multiple durations within contiguous visible runs. We construct MS-VOST by annotating 620 object histories with 2,465 states. With the encoder and temporal search fixed, D\'ej\`a Cue improves all three localization metrics over absolute scoring across eight encoder configurations, raising Top-1 temporal IoU by 0.42--1.14 percentage points. These gains persist across four text constructions, and candidate-ranking analysis shows that the vocabulary reference brings relevant intervals forward within the same search space. These results establish the supplied state vocabulary as an object-specific reference for localizing semantic states from frozen visual representations.

## Stability Bands for Feature Attribution: The Core, the Nucleolus and the Bargaining Set of Explained-Variance Games

DevFeed: [Stability Bands for Feature Attribution: The Core, the Nucleolus and the Bargaining Set of Explained-Variance Games](<https://devfeed.tech/articles/stability-bands-for-feature-attribution-the-core-the-nucleolus-and-the-bargaining-set-of-explained-variance-games-145027.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.25238>)

Author: Fernando Delbianco, Fernando Tohm\\'e

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Plover](<https://devfeed.tech/topics/plover.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [attribution](<https://devfeed.tech/tags/attribution.md>), [band](<https://devfeed.tech/tags/band.md>), [battery](<https://devfeed.tech/tags/battery.md>), [boosting](<https://devfeed.tech/tags/boosting.md>), [certificate](<https://devfeed.tech/tags/certificate.md>), [coalition](<https://devfeed.tech/tags/coalition.md>), [contest](<https://devfeed.tech/tags/contest.md>), [convex](<https://devfeed.tech/tags/convex.md>), [core](<https://devfeed.tech/tags/core.md>), [cs-gt](<https://devfeed.tech/tags/cs-gt.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [econ-em](<https://devfeed.tech/tags/econ-em.md>), [economics](<https://devfeed.tech/tags/economics.md>), [feature](<https://devfeed.tech/tags/feature.md>), [features](<https://devfeed.tech/tags/features.md>), [game](<https://devfeed.tech/tags/game.md>), [games](<https://devfeed.tech/tags/games.md>), [gradient](<https://devfeed.tech/tags/gradient.md>), [gradient-boosting](<https://devfeed.tech/tags/gradient-boosting.md>), [graph](<https://devfeed.tech/tags/graph.md>), [interaction](<https://devfeed.tech/tags/interaction.md>), [variance](<https://devfeed.tech/tags/variance.md>)

### AI overview

The paper extends feature attribution beyond single Shapley values by reporting each feature's range of stable shares over the core, or the least core when the core is empty, with the nucleolus as a reference allocation. It defines coalition worth as the variance of predictions generated by a feature set. For linear models, the game is exactly 2-additive, with pairwise interactions driven by coefficient products and feature covariances; in any 2-additive game the prenucleolus equals the Shapley value, and core non-emptiness is characterized by non-negative interaction-graph cut weights. Convex games allow closed-form bands and minimum-cut certificates. In the reported examples, a coalition of Ratings, ROM, and Battery can object to a mean-SHAP allocation in a hedonic price model. A retail-orders gradient-boosting forecaster has an empty core and strong higher-order interactions; its nucleolus differs from Shapley by up to 0.19 variance shares, and sparse third-order surrogates recover it where pairwise surrogates do not.

### Source excerpt

arXiv:2611.25238v1 Announce Type: cross Abstract: Shapley-value attributions are the default way of explaining machine-learning predictions in economics, but they report a single number per feature and say nothing about whether a coalition of features that jointly accounts for more than it receives could contest that allocation. We study feature attribution through the core, the nucleolus and the bargaining set of an explained-variance game, in which the worth of a set of features is the variance of the model's predictions that those features generate. For every feature we report a stability band (its range of shares over the core, or over the least core when the core is empty), with the nucleolus as distinguished point and a certificate that the allocation admits no justified objection. Three results organise the approach. First, for linear models the game is exactly 2-additive, with pairwise Harsanyi dividends $2\beta_i\beta_j\sigma_{ij}$, so multicollinearity is what creates coalitional structure. Second, in every 2-additive game the prenucleolus coincides with the Shapley value, the (least) core is point-symmetric around it, and the core is non-empty if and only if every cut of the interaction graph has non-negative weight. The Shapley value thus always lies in the prebargaining set, and stability adds information only through band widths and interactions of order three or more. Third, for convex games the bands have a closed form and any allocation can be certified with one minimum cut. In a hedonic price model, the usual mean-SHAP explanation admits a justified objection by the coalition {Ratings, ROM, Battery}. In a gradient-boosting forecaster of retail orders the exact game has an empty core and strong higher-order interactions; the nucleolus departs from the Shapley value by up to 0.19 variance shares, and sparse third-order surrogates recover it where pairwise ones cannot.

## The Price of an Explanation: What Demanding Verifiable Artifacts Costs, and What It Buys Back

DevFeed: [The Price of an Explanation: What Demanding Verifiable Artifacts Costs, and What It Buys Back](<https://devfeed.tech/articles/the-price-of-an-explanation-what-demanding-verifiable-artifacts-costs-and-what-it-buys-back-145115.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.25896>)

Author: Edgar A. Du\\'e\\~nez-Guzm\\'an, Eduardo Due\\~nez, Edgar Said Hernandez Sanchez, Suzanne Sadedin

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [few-shot](<https://devfeed.tech/topics/few-shot.md>), [benchmark overfitting machine learning](<https://devfeed.tech/topics/benchmark-overfitting-machine-learning.md>), [reproducibility](<https://devfeed.tech/topics/reproducibility.md>)

Tags: [agi](<https://devfeed.tech/tags/agi.md>), [artifacts](<https://devfeed.tech/tags/artifacts.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [bottleneck](<https://devfeed.tech/tags/bottleneck.md>), [budget](<https://devfeed.tech/tags/budget.md>), [costs](<https://devfeed.tech/tags/costs.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-ne](<https://devfeed.tech/tags/cs-ne.md>), [decoder](<https://devfeed.tech/tags/decoder.md>), [error-correction](<https://devfeed.tech/tags/error-correction.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [llm](<https://devfeed.tech/tags/llm.md>), [mitigation](<https://devfeed.tech/tags/mitigation.md>), [program](<https://devfeed.tech/tags/program.md>), [quantum](<https://devfeed.tech/tags/quantum.md>), [sampling](<https://devfeed.tech/tags/sampling.md>), [score](<https://devfeed.tech/tags/score.md>), [search](<https://devfeed.tech/tags/search.md>), [synthesis](<https://devfeed.tech/tags/synthesis.md>), [tasks](<https://devfeed.tech/tags/tasks.md>), [test](<https://devfeed.tech/tags/test.md>), [verification](<https://devfeed.tech/tags/verification.md>)

### AI overview

On ARC-AGI-2, requiring an LLM to produce a verifiable program reduced performance from 93/120 tasks (77%) to 68/120 (57%). The authors report that most answer-only wins remained out of reach for the program-producing configurations they tested, even when a program matched every demonstration, pointing to generalization as the key bottleneck. A short evolutionary search over two sampled programs improved on best-of-two sampled code by 9.2 ± 1.4 tasks without additional model cost. Checking programs against training examples also enabled early stopping, reducing budget use by up to 38% for best-of-n sampling and 49% for the evolved arm; answer-only runs could not stop this way. The same methodology was applied to quantum error-correction decoder synthesis, producing interpretable decoders evaluated on held-out shots against a certified panel of published decoders.

### Source excerpt

arXiv:2611.25896v1 Announce Type: cross Abstract: Generative models produce many candidate solutions but provide no intrinsic signal by which to rank them. One mitigation is to demand a \emph{verifiable artifact}: a program that can be executed against the available evidence. A common concern is that requiring executable verification incurs costs in both solution quality and computational budget. We measure these two costs on the $120$-task ARC-AGI-2 evaluation set. Demanding a verifiable artifact from an LLM reduced the score from $93$ to $68$ of $120$ tasks ($77\%$ to $57\%$). Of those $25$ tasks, $21$ were reached by no program-producing configuration we ran, at any budget, by sampling or by search; yet for $27$ of the $28$ tasks that only the answer-only arm solved, some program arm held a program that reproduced every demonstration and still failed the test. The bottleneck is obtaining an explanation that \emph{generalizes}, not finding one. A short evolutionary search over two already-sampled programs, at no additional model cost, beats best-of-$2$ sampled code by $9.2 \pm 1.4$ tasks, and over a single sampled program beats $1$-shot sampling by $9.9 \pm 1.3$. Moreover, because a program can be checked against the training examples, any program-producing protocol can terminate early once a task is explained: under the same stopping rule, best-of-$n$ sampling declines up to $38\%$ of its budget and the evolved arm up to $49\%$, whereas an answer-only baseline cannot terminate early. Applying the same methodology to decoder synthesis for quantum error correction yields interpretable decoders evaluated against a certified panel of published decoders on held-out exam shots.

## On the Impact of Feature Reduction Techniques on Adversarial Attacks

DevFeed: [On the Impact of Feature Reduction Techniques on Adversarial Attacks](<https://devfeed.tech/articles/on-the-impact-of-feature-reduction-techniques-on-adversarial-attacks-145002.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.25004>)

Author: Mohammad Meymani, Roozbeh Razavi-Far

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Adversarial attacks](<https://devfeed.tech/topics/adversarial-attacks.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Feature Engineering](<https://devfeed.tech/topics/feature-engineering.md>), [ai security](<https://devfeed.tech/topics/ai-security.md>)

Tags: [adversarial](<https://devfeed.tech/tags/adversarial.md>), [adversarial-machine-learning](<https://devfeed.tech/tags/adversarial-machine-learning.md>), [apt](<https://devfeed.tech/tags/apt.md>), [attacks](<https://devfeed.tech/tags/attacks.md>), [backdoor](<https://devfeed.tech/tags/backdoor.md>), [classifiers](<https://devfeed.tech/tags/classifiers.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [defense](<https://devfeed.tech/tags/defense.md>), [effect](<https://devfeed.tech/tags/effect.md>), [f1](<https://devfeed.tech/tags/f1.md>), [feature](<https://devfeed.tech/tags/feature.md>), [features](<https://devfeed.tech/tags/features.md>), [impact](<https://devfeed.tech/tags/impact.md>), [improvements](<https://devfeed.tech/tags/improvements.md>), [information](<https://devfeed.tech/tags/information.md>), [learning](<https://devfeed.tech/tags/learning.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [machine-learning-models](<https://devfeed.tech/tags/machine-learning-models.md>), [metrics](<https://devfeed.tech/tags/metrics.md>), [reduction](<https://devfeed.tech/tags/reduction.md>), [techniques](<https://devfeed.tech/tags/techniques.md>)

### AI overview

On the APT dataset, permutation importance ranking and univariate mutual information generally yielded the most consistent robustness improvements across the evaluated classifiers and attacks. The study compares PCA, permutation importance ranking, recursive feature elimination, and univariate mutual information across SVM, decision tree, random forest, MLP, and k-nearest neighbors models, under poisoning, backdoor, and universal adversarial perturbation attacks. It evaluates accuracy, F1-score, and mitigation score; the abstract says results for the other reduction methods vary by classifier and attack scenario.

### Source excerpt

arXiv:2611.25004v1 Announce Type: new Abstract: Adversarial attacks have posed significant threats to machine learning models by exploiting their inherent vulnerabilities. Researchers have established an interdisciplinary field called adversarial machine learning to explore these vulnerabilities and develop corresponding defense strategies. Feature reduction strategies are used to decrease the dimensionality of the data in order to reduce training time, avoid the curse of dimensionality, prevent overfitting, and mitigate redundant features. However, their effect on adversarial robustness across different classifiers and attack scenarios remains insufficiently explored. This study aims to analyze the effect of feature reduction techniques on the adversarial robustness of a diverse set of machine learning models under poisoning, backdoor, and universal adversarial perturbation attacks on the APT dataset. These models include support vector machine, decision tree, random forest, multi-layer perceptron, and k-nearest neighbors. The attained results are compared with scenarios where different feature reduction techniques are applied, including principal component analysis, permutation importance ranking, recursive feature elimination, and univariate mutual information. Metrics such as accuracy, F1-score, and mitigation score are employed to evaluate the effect of these techniques on adversarial robustness. The results demonstrate that permutation importance ranking and univariate mutual information generally provide the most consistent robustness improvements, while other methods exhibit more variable effects depending on the classifier and attack scenario.

## Backbone Matters: Benchmarking Encoder-Decoder Networks for Semantic Segmentation of Driving Scenes on BDD100K

DevFeed: [Backbone Matters: Benchmarking Encoder-Decoder Networks for Semantic Segmentation of Driving Scenes on BDD100K](<https://devfeed.tech/articles/backbone-matters-benchmarking-encoder-decoder-networks-for-semantic-segmentation-of-driving-scenes-on-bdd100k-145151.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.26191>)

Author: Ehsan Rassekh

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [autonomous vehicles](<https://devfeed.tech/topics/autonomous-vehicles.md>), [Machine Perception](<https://devfeed.tech/topics/machine-perception.md>), [image recognition](<https://devfeed.tech/topics/image-recognition.md>)

Tags: [benchmarking](<https://devfeed.tech/tags/benchmarking.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [decoder](<https://devfeed.tech/tags/decoder.md>), [driving](<https://devfeed.tech/tags/driving.md>), [encoder](<https://devfeed.tech/tags/encoder.md>), [encoder-decoder](<https://devfeed.tech/tags/encoder-decoder.md>), [networks](<https://devfeed.tech/tags/networks.md>), [semantic](<https://devfeed.tech/tags/semantic.md>)

### AI overview

On BDD100K, decoder choice strongly changes how a backbone performs: EfficientNet-B4 scores 62.6% mIoU with FPN but 42.6% with PSPNet. The study trained 52 combinations of four decoders (U-Net, FPN, LinkNet and PSPNet) and 13 backbones across six families using a common protocol. FPN with EfficientNet-B4 was best, reaching 62.6% mIoU, 89.8% accuracy and 0.092 loss; only two FPN-EfficientNet combinations exceeded the cited BDD100K benchmark of 56.9% mIoU. The best-to-worst backbone gap varied by decoder, from 12.0 points for U-Net to 42.5 for FPN. These are reported results for this dataset and protocol; the supplied text does not provide further methodological details or broader validation.

### Source excerpt

arXiv:2611.26191v1 Announce Type: cross Abstract: Semantic segmentation gives an autonomous vehicle a dense, pixel-level description of its surroundings and is a core part of scene understanding. Most segmentation networks consist of an encoder (the backbone) and a decoder, yet the backbone is often chosen without a systematic comparison. This work studies this choice on the BDD100K driving dataset. Four widely used decoders (U-Net, FPN, LinkNet and PSPNet) are combined with 13 backbones from six families (ResNet, VGG, DenseNet, Inception, MobileNet and EfficientNet), and all 52 models are trained with one common protocol. The best model, an FPN decoder with an EfficientNet-B4 backbone, reaches 62.6% mean intersection over union (mIoU), 89.8% accuracy and a loss of 0.092. Only two of the 52 models, both FPN with EfficientNet, exceed the 56.9% mIoU reported with the BDD100K benchmark. The effect of the backbone depends strongly on the decoder: the same EfficientNet-B4 gives 42.6% mIoU with PSPNet but 62.6% with FPN, and the gap between the best and the worst backbone ranges from 12.0 points (U-Net) to 42.5 points (FPN). The results and the scope of this study are discussed.

## Neural Conditional Gaussian Model: Efficient Data Assimilation with Sequential Uncertainty Quantification for Nonlinear Dynamics

DevFeed: [Neural Conditional Gaussian Model: Efficient Data Assimilation with Sequential Uncertainty Quantification for Nonlinear Dynamics](<https://devfeed.tech/articles/neural-conditional-gaussian-model-efficient-data-assimilation-with-sequential-uncertainty-quantification-for-nonlinear-dynamics-145185.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.26463>)

Author: Chuanqi Chen, Nan Chen

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Neural Network](<https://devfeed.tech/topics/neural-network.md>), [Data science and engineering](<https://devfeed.tech/topics/data-science-and-engineering.md>)

Tags: [benchmark](<https://devfeed.tech/tags/benchmark.md>), [computer-science](<https://devfeed.tech/tags/computer-science.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [data-driven](<https://devfeed.tech/tags/data-driven.md>), [decoder](<https://devfeed.tech/tags/decoder.md>), [encoder-decoder](<https://devfeed.tech/tags/encoder-decoder.md>), [forecasting](<https://devfeed.tech/tags/forecasting.md>), [gaussian](<https://devfeed.tech/tags/gaussian.md>), [math-ds](<https://devfeed.tech/tags/math-ds.md>), [model](<https://devfeed.tech/tags/model.md>), [neural](<https://devfeed.tech/tags/neural.md>), [simulation](<https://devfeed.tech/tags/simulation.md>), [uncertainty](<https://devfeed.tech/tags/uncertainty.md>), [uncertainty-quantification](<https://devfeed.tech/tags/uncertainty-quantification.md>)

### AI overview

Neural CGM learns a dynamical model by mapping physical states into a latent space with conditional Gaussian dynamics, enabling closed-form data-assimilation updates there. Nonlinear encoder-decoder mappings represent physical-space dynamics; a Gaussian decoder maps sequential latent posterior means and covariances back to physical variables using local moment propagation and learned corrections. These posterior statistics also enter a Gaussian negative log-likelihood during training, so the model learns from both estimated states and uncertainty. It can forecast and simulate without explicit governing equations. Benchmark experiments report accurate estimation and forecasting with substantially lower online assimilation cost than conventional nonlinear data-assimilation methods. The supplied abstract does not quantify these results.

### Source excerpt

arXiv:2611.26463v1 Announce Type: new Abstract: Data assimilation (DA) in partially observed nonlinear systems requires repeated estimation of both hidden states and their uncertainty. Conventional nonlinear filtering can be computationally expensive and often relies on an explicit dynamical model. In this paper, we develop the Neural Conditional Gaussian Model (Neural CGM), a data-driven dynamical model that maps unobserved physical states into a learned latent space, where the dynamics of the latent states follow a conditional Gaussian structure. This structure allows closed-form DA updates in the latent space, while nonlinear encoder-decoder mappings retain the flexibility needed to represent complex dynamics in physical space. A key feature of Neural CGM is its treatment of uncertainty in DA. The latent posterior mean and covariance are mapped to the physical variables through a Gaussian decoder that combines local moment propagation with learned nonlinear corrections. The resulting physical-space posterior statistics depend on the sequentially updated latent distribution rather than on a separate post-processing estimate. The same statistics enter a Gaussian negative log-likelihood during training, so both the posterior mean and uncertainty affect the learned model. Because Neural CGM learns the dynamical model itself, it can also be used for state forecasting and simulation without explicit knowledge of the governing equations. Numerical experiments on benchmark nonlinear dynamical systems show accurate state estimation and forecasting while substantially reducing online assimilation cost relative to conventional nonlinear DA methods. Neural CGM is therefore suited to repeated state estimation, uncertainty quantification, and prediction from partial observations.

## DL101 The Representation Capability of Neural Network Models

DevFeed: [DL101 The Representation Capability of Neural Network Models](<https://devfeed.tech/articles/dl101-the-representation-capability-of-neural-network-models-144995.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.24980>)

Author: Fernando Berzal

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Deep learning](<https://devfeed.tech/topics/deep-learning.md>), [Deep neural networks](<https://devfeed.tech/topics/deep-neural-networks.md>), [Neural Network](<https://devfeed.tech/topics/neural-network.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>)

Tags: [approximation](<https://devfeed.tech/tags/approximation.md>), [continuous](<https://devfeed.tech/tags/continuous.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-ne](<https://devfeed.tech/tags/cs-ne.md>), [deep](<https://devfeed.tech/tags/deep.md>), [deep-learning](<https://devfeed.tech/tags/deep-learning.md>), [error](<https://devfeed.tech/tags/error.md>), [gradient-descent](<https://devfeed.tech/tags/gradient-descent.md>), [guarantees](<https://devfeed.tech/tags/guarantees.md>), [neural-network](<https://devfeed.tech/tags/neural-network.md>), [regions](<https://devfeed.tech/tags/regions.md>), [relu](<https://devfeed.tech/tags/relu.md>), [stat-ml](<https://devfeed.tech/tags/stat-ml.md>), [tangent](<https://devfeed.tech/tags/tangent.md>), [training](<https://devfeed.tech/tags/training.md>), [universal-approximation](<https://devfeed.tech/tags/universal-approximation.md>), [universality](<https://devfeed.tech/tags/universality.md>)

### AI overview

Universal approximation shows that shallow neural networks with non-polynomial activations can approximate any continuous function on compact sets, but it gives no guarantees about model size, successful training, or generalization. Barron's results provide quantitative approximation rates for functions with bounded Fourier norm. Montúfar's result shows that deep networks can form exponentially more linear regions as depth increases; the excerpt states that, for the same parameter count, they are more expressive than shallow networks. These results explain representational capacity, not whether training will achieve it.

### Source excerpt

arXiv:2611.24980v1 Announce Type: cross Abstract: Universal approximation theorems (Cybenko, Funahashi, Hornik & Leshno) provide the mathematical justification for why neural networks are powerful enough to model complex, nonlinear phenomena. Shallow networks can approximate any continuous function on compact sets. Leshno's theorem clarified that non-polynomial activations are necessary for universality, which explains why ReLU, logistic, hyperbolic tangent, and other common activations work (but pure polynomials do not). However, approximation theorems provide no efficiency, training, nor generalization guarantees. They do not indicate how many neurons are needed for a given accuracy, whether gradient descent will find the right parameters, or whether networks trained on finite data will generalize well. Barron provided quantitative error bounds for particular function classes (approximation rates for functions with bounded Fourier norm), whereas Mont\'ufar was the first to show that expressivity grows exponentially with depth (in terms of the number of linear regions). Deep networks are strictly more expressive than shallow ones with the same number of parameters. Depth is fundamentally more powerful than width, providing a rigorous mathematical explanation for the use of deep learning models.

## How Network Topology and Data Heterogeneity Shape Decentralized Learning: A Spectral Analysis

DevFeed: [How Network Topology and Data Heterogeneity Shape Decentralized Learning: A Spectral Analysis](<https://devfeed.tech/articles/how-network-topology-and-data-heterogeneity-shape-decentralized-learning-a-spectral-analysis-145143.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.26104>)

Author: Luigi Palmieri, Lorenzo Valerio, Chiara Boldrini, Andrea Passarella, Marco Conti, J\\'anos Kert\\'esz

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Federated Learning](<https://devfeed.tech/topics/federated-learning.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>)

Tags: [aggregation](<https://devfeed.tech/tags/aggregation.md>), [analysis](<https://devfeed.tech/tags/analysis.md>), [communication](<https://devfeed.tech/tags/communication.md>), [connectivity](<https://devfeed.tech/tags/connectivity.md>), [consensus](<https://devfeed.tech/tags/consensus.md>), [contribution](<https://devfeed.tech/tags/contribution.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-si](<https://devfeed.tech/tags/cs-si.md>), [decentralized](<https://devfeed.tech/tags/decentralized.md>), [distributed](<https://devfeed.tech/tags/distributed.md>), [distribution](<https://devfeed.tech/tags/distribution.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [federated](<https://devfeed.tech/tags/federated.md>), [federated-learning](<https://devfeed.tech/tags/federated-learning.md>), [framework](<https://devfeed.tech/tags/framework.md>), [gradient](<https://devfeed.tech/tags/gradient.md>), [influence](<https://devfeed.tech/tags/influence.md>), [information](<https://devfeed.tech/tags/information.md>), [irregular](<https://devfeed.tech/tags/irregular.md>), [learning](<https://devfeed.tech/tags/learning.md>), [network](<https://devfeed.tech/tags/network.md>), [topology](<https://devfeed.tech/tags/topology.md>)

### AI overview

The paper separates decentralized learning's topology effects into two mechanisms: the dominant eigenmode sets the consensus trajectory and node influence, while non-dominant eigenmodes govern disagreement. Under uniform neighborhood averaging, stationary influence is proportional to node degree plus one; the authors also derive a disagreement bound involving spectral mixing and gradient heterogeneity. Experiments report nearly identical learning dynamics across regular and heterogeneous topologies with IID data. With non-IID data, network size and connectivity affect how unevenly information is learned, and the distribution of data across nodes shapes local contributions to consensus.

### Source excerpt

arXiv:2611.26104v1 Announce Type: new Abstract: Network topology influences decentralized federated learning through information mixing and the relative contribution of individual nodes. While existing analyses primarily focus on spectral mixing properties, they often assume doubly stochastic aggregation, overlooking topology-dependent influence under irregular communication networks. We develop a spectral framework that separates two complementary mechanisms: the dominant eigenmode determines the consensus trajectory and node influence, while non-dominant eigenmodes govern the propagation and attenuation of disagreement. Under uniform neighborhood averaging, we show that the stationary influence is proportional to node degree plus one and derive a bound relating model disagreement to spectral mixing and gradient heterogeneity. We then investigate how these mechanisms interact with the data distribution across nodes. Experiments across regular and heterogeneous network topologies reveal nearly identical learning dynamics under IID data, while network size and connectivity affect how unevenly distributed information is learned under non-IID data. For non-IID settings, we characterize how the distribution of data across nodes shapes the relative contribution of local information to consensus dynamics.

[Next page](<https://devfeed.tech/sources/arxiv-machine-learning.md?cursor=WyIyMDI2LTExLTIwVDA1OjAwOjAwKzAwOjAwIiwgImRmZDliYjBjLWEyZGMtNDE1My1iNmZmLTgwMTg3MmIxOTM1YiJd>)