# Model Development

A machine-learning systems-engineering concept focused on creating and refining models for specific tasks and performance goals.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## Survey-aware Machine Learning: A Guideline for Valid Population Health Inference based on Scoping Review

DevFeed: [Survey-aware Machine Learning: A Guideline for Valid Population Health Inference based on Scoping Review](<https://devfeed.tech/articles/survey-aware-machine-learning-a-guideline-for-valid-population-health-inference-based-on-scoping-review-144893.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2605.08963>)

Author: YongKyung Oh, Henry W. Zheng, Jeffrey Feng, Alex A. T. Bui

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Model Development](<https://devfeed.tech/topics/model-development.md>), [Hyperparameter optimization](<https://devfeed.tech/topics/hyperparameter-optimization.md>)

Tags: [cross-validation](<https://devfeed.tech/tags/cross-validation.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [deployment](<https://devfeed.tech/tags/deployment.md>), [design](<https://devfeed.tech/tags/design.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [fairness](<https://devfeed.tech/tags/fairness.md>), [guidance](<https://devfeed.tech/tags/guidance.md>), [health](<https://devfeed.tech/tags/health.md>), [inference](<https://devfeed.tech/tags/inference.md>), [lifecycle](<https://devfeed.tech/tags/lifecycle.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [metadata](<https://devfeed.tech/tags/metadata.md>), [ml](<https://devfeed.tech/tags/ml.md>), [model-training](<https://devfeed.tech/tags/model-training.md>), [models](<https://devfeed.tech/tags/models.md>), [nutrition](<https://devfeed.tech/tags/nutrition.md>), [objectives](<https://devfeed.tech/tags/objectives.md>), [performance](<https://devfeed.tech/tags/performance.md>), [performance-evaluation](<https://devfeed.tech/tags/performance-evaluation.md>), [review](<https://devfeed.tech/tags/review.md>), [stat-ml](<https://devfeed.tech/tags/stat-ml.md>), [survey](<https://devfeed.tech/tags/survey.md>)

### AI overview

Standard ML evaluation on complex health surveys can produce biased estimates, understate uncertainty, and miss population-level disparities when it ignores sampling units, stratification, and weights. The paper proposes Survey-aware Machine Learning (SaML), a nine-step guideline for incorporating survey design metadata across the ML lifecycle. Its scoping review of 16 methodological papers summarizes weighted training, design-based cross-validation, and survey-adjusted performance evaluation, while identifying gaps in hyperparameter tuning and deployment. The guideline offers task-specific direction on which steps are needed for different analytical objectives.

### Source excerpt

arXiv:2605.08963v2 Announce Type: replace-cross Abstract: Machine Learning (ML) models trained on complex health surveys such as the National Health and Nutrition Examination Survey (NHANES) often ignore primary sampling units, stratification variables, and sampling weights. This practice violates the independence assumptions of standard evaluation methods. As a result, estimates become biased, uncertainty is underestimated, and fairness assessments fail to reflect population-level disparities. We propose Survey-aware Machine Learning (SaML), a nine-step guideline that incorporates survey design metadata across the ML lifecycle. Through a scoping review of 16 methodological papers, we summarize existing work on weighted model training, design-based cross-validation, and survey-adjusted performance evaluation. We also identify gaps in hyperparameter tuning and deployment. We provide task-specific guidance that clarifies which steps are required for different analytical objectives. SaML provides a checklist for valid population inference from survey data.

## Approximate Adjoint Models for Memory and Compute-Efficient Online Learning of Hybrid Models

DevFeed: [Approximate Adjoint Models for Memory and Compute-Efficient Online Learning of Hybrid Models](<https://devfeed.tech/articles/approximate-adjoint-models-for-memory-and-compute-efficient-online-learning-of-hybrid-models-145140.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.26095>)

Author: Emilio Gonz\\'alez-Zamora, Pierre Tandeo, Laurent Debreu, Said Ouala

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Model Development](<https://devfeed.tech/topics/model-development.md>)

Tags: [approximation](<https://devfeed.tech/tags/approximation.md>), [backpropagation](<https://devfeed.tech/tags/backpropagation.md>), [bptt](<https://devfeed.tech/tags/bptt.md>), [closure](<https://devfeed.tech/tags/closure.md>), [compute](<https://devfeed.tech/tags/compute.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [hybrid](<https://devfeed.tech/tags/hybrid.md>), [learning](<https://devfeed.tech/tags/learning.md>), [math-oc](<https://devfeed.tech/tags/math-oc.md>), [memory](<https://devfeed.tech/tags/memory.md>), [model-training](<https://devfeed.tech/tags/model-training.md>), [ocean](<https://devfeed.tech/tags/ocean.md>), [online](<https://devfeed.tech/tags/online.md>), [online-learning](<https://devfeed.tech/tags/online-learning.md>), [physics-ao-ph](<https://devfeed.tech/tags/physics-ao-ph.md>)

### AI overview

The proposed methods reduce the memory and compute costs of online hybrid-model training while preserving adjoint recurrence across the full trajectory. They compute exact pullbacks only at sparse anchor states; between anchors, Zero-order propagates sensitivities through the identity, while Pullback Reuse uses a frozen pullback. Reusing the pullback also lets the parameter-gradient sum factorize to one evaluation per chunk instead of one per time step. Across an incomplete Lorenz-63 model, a two-dimensional quasi-geostrophic turbulence closure, and bias correction in a multilayer quasi-geostrophic ocean model, peak memory fell by up to an order of magnitude and runtime by up to fourfold. Pullback Reuse stayed closest to the exact gradient direction, and both methods supported training at comparable accuracy. These are abstract-level reported results; no further experimental details are provided here.

### Source excerpt

arXiv:2611.26095v1 Announce Type: new Abstract: Differentiable programming enables online calibration of numerical models by minimizing errors accumulated along simulated trajectories. This includes estimating physical parameters and training neural sub-models in hybrid configurations. Gradients are typically computed using adjoint methods, or backpropagation through time (BPTT), whose computational and memory costs grow with the length of the training window. This limits the duration and resolution of hybrid-model training, a major bottleneck for realistic geophysical applications. Approaches like truncated BPTT reduces memory use by discarding sensitivities across truncation boundaries, introducing gradient bias and neglecting long-range dependencies. We propose a class of gradient approximation methods that preserve the adjoint recurrence over the full trajectory while approximating only the pullback operators within it. Exact pullbacks are computed by automatic differentiation at sparse anchor states only. Between anchors, sensitivities propagate either through the identity (Zero-order) or through a reused, frozen pullback (Pullback Reuse). This approximation makes the parameter pullback constant within each chunk, allowing the gradient sum to factorize so that a single evaluation per chunk replaces one per time step. We evaluate both on three systems of increasing complexity, an incomplete Lorenz-63 model, a large-eddy closure for two-dimensional quasi-geostrophic turbulence, and bias correction in a multilayer quasi-geostrophic ocean model. Peak memory falls by up to an order of magnitude and runtime up to fourfold, with Pullback Reuse staying closest to the exact gradient direction. In all three cases, approximate gradients support hybrid-model training at comparable accuracy, showing that effective online learning does not require exact gradients.

## SPACE: Source-free Proxy Anchor Concept Erasure for MLLMs

DevFeed: [SPACE: Source-free Proxy Anchor Concept Erasure for MLLMs](<https://devfeed.tech/articles/space-source-free-proxy-anchor-concept-erasure-for-mllms-144904.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2606.09868>)

Author: Zhijing Zhang, Jiaqi Ding, Qianshan Wei, Nan Zhou, Jiaqi Li, Yongliang Wu, Tongxin Zhu, Xiaolin Fang

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Model Development](<https://devfeed.tech/topics/model-development.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Embeddings](<https://devfeed.tech/topics/embeddings.md>)

Tags: [code](<https://devfeed.tech/tags/code.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [data-retention](<https://devfeed.tech/tags/data-retention.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [entropy](<https://devfeed.tech/tags/entropy.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [knowledge](<https://devfeed.tech/tags/knowledge.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [machine-unlearning](<https://devfeed.tech/tags/machine-unlearning.md>), [multimodal](<https://devfeed.tech/tags/multimodal.md>), [privacy](<https://devfeed.tech/tags/privacy.md>), [proxy](<https://devfeed.tech/tags/proxy.md>), [regulatory](<https://devfeed.tech/tags/regulatory.md>)

### AI overview

SPACE is a source-free machine unlearning framework for multimodal large language models that aims to erase target concepts without access to their visual data. Text-Guided Proxy Anchor Selection retrieves semantically aligned proxy anchors from the shared feature space; Dual-Constraint Semantic Isolation then optimizes those anchors to indirectly erase the concepts while confining updates to the null space of retained knowledge. The authors state that this bounds perturbations to retained knowledge and maximizes feature spectral entropy. Experiments across six datasets reportedly achieve performance comparable to state-of-the-art methods that use target data.

### Source excerpt

arXiv:2606.09868v2 Announce Type: replace Abstract: As Multimodal Large Language Models (MLLMs) face growing privacy risks and regulatory constraints, machine unlearning (MU) has emerged as a crucial solution for removing sensitive data while preserving model performance. However, existing MU methods typically rely on visual data of the target concepts, which is often unavailable due to strict data retention policies, thus creating a demand for source-free unlearning approaches that operate without access to the target data. In this work, we propose Source-free Proxy Anchor Concept Erasure (SPACE), the first source-free unlearning framework specialized for MLLMs. SPACE consists of two stages: (1) Text-Guided Proxy Anchor Selection (TPAS), which retrieves semantically aligned proxy anchors from the shared feature space. (2) Dual-Constraint Semantic Isolation (DCSI), which optimizes these anchors to indirectly erase target concepts. DCSI confines updates to the null space of retained knowledge, ensuring structural integrity. We theoretically prove that SPACE strictly bounds the perturbation on retained knowledge and maximizes feature spectral entropy, thereby maintaining the model's performance. Furthermore, extensive experiments across six datasets show that SPACE achieves performance comparable to that of state-of-the-art data-dependent methods, validating its effectiveness in source-free MU scenarios. The source code will be released.

## Benchmarking Robust Aggregation in Decentralized Gradient Marketplaces

DevFeed: [Benchmarking Robust Aggregation in Decentralized Gradient Marketplaces](<https://devfeed.tech/articles/benchmarking-robust-aggregation-in-decentralized-gradient-marketplaces-144856.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2509.05833>)

Author: Zeyu Song, Sainyam Galhotra, Shagufta Mehnaz

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Federated Learning](<https://devfeed.tech/topics/federated-learning.md>), [Ground truth / benchmark quality](<https://devfeed.tech/topics/ground-truth-benchmark-quality.md>), [Model Development](<https://devfeed.tech/topics/model-development.md>)

Tags: [adverse-selection](<https://devfeed.tech/tags/adverse-selection.md>), [aggregation](<https://devfeed.tech/tags/aggregation.md>), [benchmarking](<https://devfeed.tech/tags/benchmarking.md>), [contamination](<https://devfeed.tech/tags/contamination.md>), [contributions](<https://devfeed.tech/tags/contributions.md>), [cs-gt](<https://devfeed.tech/tags/cs-gt.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [data](<https://devfeed.tech/tags/data.md>), [data-marketplace](<https://devfeed.tech/tags/data-marketplace.md>), [data-products](<https://devfeed.tech/tags/data-products.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [decentralized](<https://devfeed.tech/tags/decentralized.md>), [differential-privacy](<https://devfeed.tech/tags/differential-privacy.md>), [distributed](<https://devfeed.tech/tags/distributed.md>), [gradient](<https://devfeed.tech/tags/gradient.md>), [malicious](<https://devfeed.tech/tags/malicious.md>), [marketplace](<https://devfeed.tech/tags/marketplace.md>), [marketplaces](<https://devfeed.tech/tags/marketplaces.md>), [mechanism](<https://devfeed.tech/tags/mechanism.md>), [model](<https://devfeed.tech/tags/model.md>), [model-performance](<https://devfeed.tech/tags/model-performance.md>), [robust](<https://devfeed.tech/tags/robust.md>)

### AI overview

The benchmark finds that aggregation mechanisms can protect model performance while misallocating payment in gradient data marketplaces. Across a standard baseline and 11 aggregation mechanisms on five vision, text, and tabular datasets, honest high-variance updates were often rejected as outliers while strategically homogeneous malicious updates were accepted and rewarded. The authors report that this adverse selection persists with continuous weight-proportional payment, compounds over repeated rounds as underpaid honest sellers exit, and worsens when differential-privacy noise makes honest updates look contaminated. The framework evaluates mechanism integrity--whether selection and payment reward utility--separately from model integrity.

### Source excerpt

arXiv:2509.05833v2 Announce Type: replace Abstract: Institutions that cannot centralize or sell raw data increasingly exchange model updates instead: cross-silo federated learning is already in production among hospital networks and pharmaceutical consortia. A growing line of research targets the next step: marketplaces in which gradient updates are exchanged as paid, query-specific data products, though none is yet deployed at scale. We formalize this emerging paradigm as a Gradient Data Marketplace (GDM), in which data consumers issue learning tasks and distributed participants contribute query-specific gradient updates. Federated learning asks whether the model survives an attack; a marketplace must also ask who got paid. Existing protocols ensure model-level robustness but fail to capture whether high-quality data is correctly identified and rewarded. In a GDM, selection determines payment, incentivizing participants to optimize for inclusion rather than utility. We introduce a modular benchmarking framework that separates mechanism integrity (whether selection and payment reward true utility) from model integrity, and evaluate a standard baseline alongside 11 widely used aggregation mechanisms on five real-world vision, text, and tabular datasets. Our results reveal consistent adverse selection: honest high-variance contributions are rejected as outliers while strategically homogeneous malicious contributions are accepted and rewarded, even when model performance remains stable. These failures are not artifacts of our accounting: they persist under continuous, weight-proportional payment, compound across repeated market rounds as underpaid honest sellers exit, and worsen when honest sellers add differential-privacy noise, which the filters misread as contamination. Our benchmark provides a foundation for designing incentive-aware, auditable data selection mechanisms in emerging data marketplaces.

## Gradients for Interventions and Activation Values for Detection: Targeted Feature Learning in Language Models

DevFeed: [Gradients for Interventions and Activation Values for Detection: Targeted Feature Learning in Language Models](<https://devfeed.tech/articles/gradients-for-interventions-and-activation-values-for-detection-targeted-feature-learning-in-language-models-144935.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2609.32355>)

Author: Jonathan Drechsel, Steffen Herbold

Published: 2026-11-20T05:00:00Z

Content type: comparison

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Model Development](<https://devfeed.tech/topics/model-development.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [LLM Techniques](<https://devfeed.tech/topics/llm-techniques.md>)

Tags: [activation](<https://devfeed.tech/tags/activation.md>), [autoencoders](<https://devfeed.tech/tags/autoencoders.md>), [compare](<https://devfeed.tech/tags/compare.md>), [concepts](<https://devfeed.tech/tags/concepts.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [detection](<https://devfeed.tech/tags/detection.md>), [feature](<https://devfeed.tech/tags/feature.md>), [gradient](<https://devfeed.tech/tags/gradient.md>), [interpretability](<https://devfeed.tech/tags/interpretability.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [learning](<https://devfeed.tech/tags/learning.md>), [models](<https://devfeed.tech/tags/models.md>), [performance](<https://devfeed.tech/tags/performance.md>), [results](<https://devfeed.tech/tags/results.md>), [signal](<https://devfeed.tech/tags/signal.md>), [steering](<https://devfeed.tech/tags/steering.md>), [study](<https://devfeed.tech/tags/study.md>), [tasks](<https://devfeed.tech/tags/tasks.md>), [values](<https://devfeed.tech/tags/values.md>)

### AI overview

The abstract reports that targeted feature quality depends on both the model signal and estimator, and that detection and intervention measure complementary properties. In a comparison across 15 tasks and ten language models, contrastive activation value methods performed strongest for detection, while gradient-based methods--especially contrastive means--performed strongest for intervention. Within the contrastive mean methods, activation gradients substantially improved intervention over activation values without requiring parameter-space intervention. The study compares six targeted methods spanning activation values, activation gradients, and parameter gradients, paired with either a contrastive mean or a learned one-dimensional encoder-decoder, against pretrained Sparse Autoencoders.

### Source excerpt

arXiv:2609.32355v2 Announce Type: replace-cross Abstract: Model-internal features can be studied through both their ability to identify a specified concept and their causal effect when manipulated, e.g., through steering or weight editing. A prominent approach to feature learning is Sparse Autoencoders (SAEs), which learn broad feature dictionaries whose relation to particular concepts is typically identified post hoc. However, many interpretability questions are instead hypothesis-driven and concern a concept specified in advance. We study this setting as targeted feature learning, where a single feature is constructed for such a predefined concept. We present a controlled comparison across three model signals (activation values, activation gradients, and parameter gradients) and two estimators (contrastive mean and a learned one-dimensional encoder-decoder), yielding six targeted methods, with CAA and GRADIEND as existing instances and four new methods covering the remaining combinations. We compare these methods against pretrained SAEs across 15 tasks and ten language models, evaluating both detection and causal intervention. Across models, the strongest detection performance is achieved by contrastive activation value methods, whereas the strongest intervention performance is achieved by gradient signals, most strongly with a contrastive mean. Notably, within the contrastive mean family, activation gradients substantially improve intervention over activation values without requiring parameter space interventions. Overall, our results show that targeted feature quality depends jointly on the model signal and estimator, with detection and intervention capturing complementary properties.

## UniCom: A Unified Framework for Community Search and Detection via Transfer Learning

DevFeed: [UniCom: A Unified Framework for Community Search and Detection via Transfer Learning](<https://devfeed.tech/articles/unicom-a-unified-framework-for-community-search-and-detection-via-transfer-learning-145729.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2512.02460>)

Author: Yifan Zhu, Hanchen Wang, Wenjie Zhang, Alexander Zhou, Ying Zhang

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Databases](<https://devfeed.tech/sources/arxiv-databases.md>)

Topics: [Model Development](<https://devfeed.tech/topics/model-development.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>)

Tags: [community](<https://devfeed.tech/tags/community.md>), [cs-db](<https://devfeed.tech/tags/cs-db.md>), [cs-si](<https://devfeed.tech/tags/cs-si.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [detection](<https://devfeed.tech/tags/detection.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [framework](<https://devfeed.tech/tags/framework.md>), [graph](<https://devfeed.tech/tags/graph.md>), [knowledge](<https://devfeed.tech/tags/knowledge.md>), [learning](<https://devfeed.tech/tags/learning.md>), [real-world](<https://devfeed.tech/tags/real-world.md>), [runtime](<https://devfeed.tech/tags/runtime.md>), [search](<https://devfeed.tech/tags/search.md>), [transfer-learning](<https://devfeed.tech/tags/transfer-learning.md>)

### AI overview

UniCom combines community search, which finds a community for a query, and community detection, which partitions a graph, in one transfer-learning framework. Its Universal Graph Learning stage distills local and global community knowledge from multiple source domains; Domain-aware Specialization then adapts to unseen graphs and tasks on the fly, avoiding costly retraining while keeping the model compact. The abstract reports experiments on 16 datasets against 22 baselines, claiming UniCom outperforms all state-of-the-art baselines across both tasks while maintaining runtime efficiency. The supplied evidence does not give performance details or explain the adaptation procedure.

### Source excerpt

arXiv:2512.02460v2 Announce Type: replace-cross Abstract: Searching and detecting communities in real-world graphs underpins a wide range of applications. Specifically, community search identifies a community for a given query, whereas community detection partitions the entire graph into multiple communities. Although they target different community analysis objectives, both tasks aim to uncover cohesive community structures and are complementary in real-world applications. Despite this connection, current learning-based solutions generally regard them as separate problems, necessitating task- and dataset-specific retraining. This separated learning paradigm limits the applicability and generalization ability of the existing models. To mitigate this limitation, we propose UniCom, a unified framework to solve both community search and detection tasks. UniCom centers on a Universal Graph Learning (UGL) stage that distills local and global community-aware knowledge from multiple source domains. Following UGL, the Domain-aware Specialization (DAS) stage supports on-the-fly adaptation to unseen graphs and tasks through transfer learning, avoiding costly retraining while keeping the model compact. Extensive experiments on both tasks across 16 datasets and 22 baselines have been conducted to ensure a comprehensive, rigorous, and fair evaluation. UniCom consistently outperforms all state-of-the-art baselines across all tasks while maintaining runtime efficiency.

## SKILL.PNG: Learning Agent Skills in Pixel Space

DevFeed: [SKILL.PNG: Learning Agent Skills in Pixel Space](<https://devfeed.tech/articles/skill-png-learning-agent-skills-in-pixel-space-145160.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.26296>)

Author: Maximilian Seeth, Chengzhi Martin Hu, Marie Kempkes, Paul R\\"ottger, Leo Schwinn

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Agent Skills](<https://devfeed.tech/topics/agent-skills.md>), [vlm](<https://devfeed.tech/topics/vlm.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Model Development](<https://devfeed.tech/topics/model-development.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-agents](<https://devfeed.tech/tags/ai-agents.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [gradient-descent](<https://devfeed.tech/tags/gradient-descent.md>), [hybrid](<https://devfeed.tech/tags/hybrid.md>), [image](<https://devfeed.tech/tags/image.md>), [instructions](<https://devfeed.tech/tags/instructions.md>), [language](<https://devfeed.tech/tags/language.md>), [learning](<https://devfeed.tech/tags/learning.md>), [llms](<https://devfeed.tech/tags/llms.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [optimized](<https://devfeed.tech/tags/optimized.md>), [performance](<https://devfeed.tech/tags/performance.md>), [pixel](<https://devfeed.tech/tags/pixel.md>), [pixels](<https://devfeed.tech/tags/pixels.md>), [skill](<https://devfeed.tech/tags/skill.md>), [vision-language-models](<https://devfeed.tech/tags/vision-language-models.md>), [vision-language-models-vlms](<https://devfeed.tech/tags/vision-language-models-vlms.md>), [vlms](<https://devfeed.tech/tags/vlms.md>)

### AI overview

Skill.png represents agent skills as images whose pixels are optimized by gradient descent through the vision encoder of a frozen vision-language model (VLM). The authors report that these image skills beat frontier-LLM-optimized textual skills in 9 of 16 settings across four VLMs and four benchmarks, using 64-256 image tokens at inference versus about 900-2,000 text tokens. Jointly optimized text and image skills further improve performance. An image optimized across an ensemble can transfer among its VLMs, while the reported compatibility claim is for VLMs. The abstract describes the evaluation results but does not provide per-setting details.

### Source excerpt

arXiv:2611.26296v1 Announce Type: new Abstract: AI agents are commonly specialized for new tasks and domains using skills, i.e. natural language instructions placed in their context. However, the discrete nature of textual skills prevents continuous optimization. Gradient-based approaches such as finetuning, on the other hand, are typically model-specific, preventing skill transfer across different agents. Since many agents today are built on vision-language models (VLMs), we can address these limitations by introducing Skill.png: a method for obtaining skill-like image inputs whose pixels are optimized with gradient descent through the vision encoder of a frozen VLM. Image skills are compatible with any VLM, and a single image can be optimized jointly for an ensemble of models. We show that Skill.png significantly outperforms state-of-the-art textual skills optimized using frontier LLMs in 9 of 16 settings (across 4 VLMs and 4 benchmarks) while requiring only 64 to 256 image tokens instead of roughly 900 to 2000 text tokens at inference time. Hybrid skills, i.e. text and image skills jointly optimized, further improve agent performance. Moreover, an image optimized with an ensemble of VLMs can successfully transfer across the models in the ensemble. Skill.png thus presents the first agent skill that can be optimized with gradient descent while retaining transfer benefits of text skills. Our code is available at: https://github.com/omseeth/skill.png

## From field-scale to large-scale spectral libraries: Tabular foundation models in soil spectroscopy

DevFeed: [From field-scale to large-scale spectral libraries: Tabular foundation models in soil spectroscopy](<https://devfeed.tech/articles/from-field-scale-to-large-scale-spectral-libraries-tabular-foundation-models-in-soil-spectroscopy-144915.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2608.00608>)

Author: Viacheslav Barkov, Jonas Schmidinger, Robin Gebbers, Martin Atzmueller

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Model Development](<https://devfeed.tech/topics/model-development.md>), [AI Foundation Models](<https://devfeed.tech/topics/ai-foundation-models.md>), [Feature Engineering](<https://devfeed.tech/topics/feature-engineering.md>), [Neural Network](<https://devfeed.tech/topics/neural-network.md>), [Big data infrastructure](<https://devfeed.tech/topics/big-data-infrastructure.md>)

Tags: [benchmark](<https://devfeed.tech/tags/benchmark.md>), [chemometrics](<https://devfeed.tech/tags/chemometrics.md>), [cnn](<https://devfeed.tech/tags/cnn.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [digital](<https://devfeed.tech/tags/digital.md>), [evidence-based](<https://devfeed.tech/tags/evidence-based.md>), [foundation-models](<https://devfeed.tech/tags/foundation-models.md>), [global](<https://devfeed.tech/tags/global.md>), [in-context-learning](<https://devfeed.tech/tags/in-context-learning.md>), [large](<https://devfeed.tech/tags/large.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [model-selection](<https://devfeed.tech/tags/model-selection.md>), [performance](<https://devfeed.tech/tags/performance.md>), [principal-component-analysis](<https://devfeed.tech/tags/principal-component-analysis.md>), [reduction](<https://devfeed.tech/tags/reduction.md>), [regression](<https://devfeed.tech/tags/regression.md>), [spectroscopy](<https://devfeed.tech/tags/spectroscopy.md>), [tabpfn](<https://devfeed.tech/tags/tabpfn.md>)

### AI overview

Across 85 soil property prediction tasks spanning field-scale mapping and a global spectral library, TabPFN performed best overall among the compared models. It outperformed classical baselines even when applied directly to full spectra, so dimensionality reduction was not required for strong results. PLS latent variables improved performance for all models; combined with TabPFN, they reduced RMSE versus the strongest classical baseline on 44 of 46 large-scale library tasks, with a median reduction of 14%. The abstract reports results for vis-NIR and MIR spectroscopy and datasets including tens of thousands of samples.

### Source excerpt

arXiv:2608.00608v2 Announce Type: replace Abstract: Visible and near-infrared (vis-NIR) and mid-infrared (MIR) spectroscopy enable rapid, cost-effective prediction of soil properties. Yet, translating high-dimensional, highly collinear spectra into accurate soil property predictions remains challenging, particularly when employing machine learning. We systematically investigated regression models and dimensionality reduction approaches for spectroscopic modeling across 85 soil property prediction tasks from open benchmark datasets in pedometrics, spanning field-scale digital soil mapping and a global soil spectral library. We compared an in-context learning tabular foundation model (TabPFN), a convolutional neural network (CNN), rule-based regression (Cubist), Random Forest, and partial least squares regression (PLSR) using full spectra as well as features derived from principal component analysis (PCA) and partial least squares (PLS) latent variables. TabPFN consistently delivered the best overall performance across spectral regions, properties, and scales, including large spectral library tasks with tens of thousands of soil samples. Notably, TabPFN applied directly to full spectra already surpassed all classical baselines, showing that explicit dimensionality reduction is not strictly required for strong performance. Further improvements were achieved through PLS, which proved to be an effective dimensionality reduction strategy for all models. Combining PLS latent variables with TabPFN yielded the best predictions overall, lowering the root mean squared error (RMSE) relative to the strongest classical baseline on 44 of the 46 large-scale spectral library tasks, by a median of 14%. Our findings provide evidence-based guidance for spectroscopic calibration model selection across operational scales, demonstrating that the long-standing advantages of PLSR and modern tabular foundation models complement each other in chemometrics.

## UniReFP: Robust Unified Fingerprinting for Vision Models against Cross-Task Repurposing Attacks

DevFeed: [UniReFP: Robust Unified Fingerprinting for Vision Models against Cross-Task Repurposing Attacks](<https://devfeed.tech/articles/unirefp-robust-unified-fingerprinting-for-vision-models-against-cross-task-repurposing-attacks-146035.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.25555>)

Author: Ziye Geng, Guang Yang, Yihang Chen, Lingfeng Yao, Xing Shi, Miao Pan, Xin Fu, Changqing Luo

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Cryptography and Security](<https://devfeed.tech/sources/arxiv-cryptography-and-security.md>)

Topics: [object-detection](<https://devfeed.tech/topics/object-detection.md>), [Model Development](<https://devfeed.tech/topics/model-development.md>), [Algorithms](<https://devfeed.tech/topics/algorithms.md>)

Tags: [attacks](<https://devfeed.tech/tags/attacks.md>), [classifier](<https://devfeed.tech/tags/classifier.md>), [consistency](<https://devfeed.tech/tags/consistency.md>), [cs-cr](<https://devfeed.tech/tags/cs-cr.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [formats](<https://devfeed.tech/tags/formats.md>), [framework](<https://devfeed.tech/tags/framework.md>), [intermediate-representations](<https://devfeed.tech/tags/intermediate-representations.md>), [models](<https://devfeed.tech/tags/models.md>), [ownership](<https://devfeed.tech/tags/ownership.md>), [robust](<https://devfeed.tech/tags/robust.md>), [semantic](<https://devfeed.tech/tags/semantic.md>), [space](<https://devfeed.tech/tags/space.md>), [tasks](<https://devfeed.tech/tags/tasks.md>), [verification](<https://devfeed.tech/tags/verification.md>), [vision](<https://devfeed.tech/tags/vision.md>)

### AI overview

The abstract identifies cross-task repurposing--changing a vision model's task and output format--as an attack that can make existing fingerprints inapplicable. UniReFP generates grouped fingerprints using a repurposed surrogate classifier, encoding ownership evidence in a shared class-level semantic space. At verification, task-aware projection maps suspect outputs to semantic-presence vectors; ownership is assessed by consistency within fingerprint groups. The abstract reports superior performance under cross-task repurposing, other commonly studied model modifications, and combinations of these attacks. It provides no experimental details or quantitative results.

### Source excerpt

arXiv:2611.25555v1 Announce Type: cross Abstract: We identify, for the first time, a new model modification attack--the cross-task model repurposing attack--that can render fingerprints generated by existing model fingerprinting approaches inapplicable. To address this challenge, we propose UniReFP--a novel unified model fingerprinting framework applicable to diverse vision models across different tasks--that utilizes ownership evidence in a shared class-level semantic space rather than original task-specific output space, enabling effective ownership verification when a protected model is repurposed across tasks with heterogeneous output formats. Specifically, UniReFP constructs a surrogate classifier by repurposing the protected model, enabling fingerprints to be generated independently of the original task-dependent output space of the protected model. Moreover, it optimizes grouped fingerprints to encode ownership evidence: fingerprints within the same group are encouraged to induce consistent semantic responses and intermediate representations on the surrogate classifier, while producing dispersed responses on independently trained reference classifiers. During verification, UniReFP uses a task-aware semantic projection mechanism to map heterogeneous suspect-model outputs into unified semantic-presence vectors, and determines ownership by measuring inner-group consistency. Extensive experimental results validate the effectiveness of UniReFP, consistently showing superior performance under cross-task repurposing attacks, commonly studied model modification attacks, and even their combinations.

## Multi-institutional grading of Barrett's esophagus dysplasia without stain normalization using a pathology foundation model and multiple instance learning

DevFeed: [Multi-institutional grading of Barrett's esophagus dysplasia without stain normalization using a pathology foundation model and multiple instance learning](<https://devfeed.tech/articles/multi-institutional-grading-of-barrett-s-esophagus-dysplasia-without-stain-normalization-using-a-pathology-foundation-model-and-multiple-instance-learning-145686.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.26562>)

Author: Amirali Khosravi, Jaidip M. Jagtap, D. Chamil Codipilly, Catherine E. Hagen, Jason T. Lewis, Rickey E. Carter, Shahriar Faghani, Vani J. A. Konda, Fouad Otaki, Shajan Peter, Nicholas Shaheen, Sachin W

Published: 2026-11-20T05:00:00Z

Content type: news

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [AI Foundation Models](<https://devfeed.tech/topics/ai-foundation-models.md>), [Neural Network](<https://devfeed.tech/topics/neural-network.md>), [Model Development](<https://devfeed.tech/topics/model-development.md>)

Tags: [based](<https://devfeed.tech/tags/based.md>), [classification](<https://devfeed.tech/tags/classification.md>), [classifiers](<https://devfeed.tech/tags/classifiers.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [deep-learning](<https://devfeed.tech/tags/deep-learning.md>), [diagnostic](<https://devfeed.tech/tags/diagnostic.md>), [foundation-model](<https://devfeed.tech/tags/foundation-model.md>), [institutions](<https://devfeed.tech/tags/institutions.md>), [learning](<https://devfeed.tech/tags/learning.md>), [patch](<https://devfeed.tech/tags/patch.md>), [performance](<https://devfeed.tech/tags/performance.md>), [study](<https://devfeed.tech/tags/study.md>)

### AI overview

In a retrospective study of 969 H&E whole-slide images from four institutions, a foundation-model system graded Barrett's esophagus as non-dysplastic, low-grade, or high-grade dysplasia. Expert-annotated patch classifiers first identified relevant regions; attribute-aware multiple instance learning then produced slide-level classifications. On a held-out set of 130 slides, macro-averaged AUC was 0.96 (95% CI 0.93-0.98). For binary dysplasia detection, sensitivity was 98% and specificity 81%; no high-grade case was labeled non-dysplastic. Performance was consistent across institutions without stain normalization, suggesting the approach may reduce site-specific preprocessing needs. These are retrospective diagnostic-accuracy results, not evidence of clinical outcome benefit.

### Source excerpt

arXiv:2611.26562v1 Announce Type: cross Abstract: Barrett's esophagus dysplasia grading suffers substantial inter-observer variability, particularly for low-grade dysplasia, complicating clinical decision-making. We conducted a retrospective multi-institutional diagnostic accuracy study of a foundation model-based deep learning system for dysplasia grading, using 969 H\&E-stained whole slide images from four institutions. Patch-level classifiers trained on expert annotations identified diagnostically relevant regions, followed by attribute-aware multiple instance learning for slide-level classification as non-dysplastic Barrett's esophagus (NDBE), low-grade dysplasia, or high-grade dysplasia. On a held-out test set (n = 130), macro-averaged AUC was 0.96 (95\% CI 0.93--0.98). For binary dysplasia detection, sensitivity was 98\% (80/82) and specificity 81\% (39/48), with a negative likelihood ratio of 0.030 and quadratic-weighted \kap{} of 0.83 against expert consensus. No high-grade dysplasia case was misclassified as NDBE. Performance was consistent across institutions without stain normalization, indicating that stain-robust foundation-model representations may remove the site-specific preprocessing that has constrained prior systems to institution-specific deployment.

## Drifting, Discriminating, and Directioning for Action Coordination in Higher-dimensional Action Space

DevFeed: [Drifting, Discriminating, and Directioning for Action Coordination in Higher-dimensional Action Space](<https://devfeed.tech/articles/drifting-discriminating-and-directioning-for-action-coordination-in-higher-dimensional-action-space-145048.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.25385>)

Author: Taisuke Kobayashi

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Model Development](<https://devfeed.tech/topics/model-development.md>), [World models](<https://devfeed.tech/topics/world-models.md>)

Tags: [actions](<https://devfeed.tech/tags/actions.md>), [adversarial](<https://devfeed.tech/tags/adversarial.md>), [algorithm](<https://devfeed.tech/tags/algorithm.md>), [control](<https://devfeed.tech/tags/control.md>), [coupling](<https://devfeed.tech/tags/coupling.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [policy](<https://devfeed.tech/tags/policy.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [rl](<https://devfeed.tech/tags/rl.md>), [sampling](<https://devfeed.tech/tags/sampling.md>), [spaces](<https://devfeed.tech/tags/spaces.md>), [training](<https://devfeed.tech/tags/training.md>), [variance](<https://devfeed.tech/tags/variance.md>)

### AI overview

The method combines a drifting policy model with three training and action-selection steps to capture coordination in high-dimensional reinforcement-learning actions. It co-optimizes the model's source distribution as a diagonal policy so the drifting model can represent coordination as residuals, uses adversarial training with a discriminator before and after drifting to promote that coordination, and applies best-of-N sampling during rollouts to guide action selection. Across three locomotion tasks, it reportedly improves sample efficiency and final control performance over conventional diagonal distribution modeling. The analysis attributes the gains to reduced variance in some action dimensions, effectively coupling them with other actions.

### Source excerpt

arXiv:2611.25385v1 Announce Type: new Abstract: This paper proposes a lightweight policy model to mitigate the difficulty of handling higher-dimensional action spaces in reinforcement learning (RL), alongside an algorithm to effectively train it. To address this, we focus on drifting models, which can implicitly represent complex distributions and infer actions in a single forward pass. However, while drifting models can be straightforwardly trained via backpropagation of the Q-gradient, this alone cannot be expected to appropriately capture the desired action coordination. Therefore, this paper constructs a three-step learning algorithm to encourage the action coordination. First, the source distribution of the drifting model is co-optimized as a diagonal policy model. This conditions the drifting model to attach the coordination, which cannot be expressed by a diagonal distribution, as residuals. Second, a discriminator model before and after drifting is introduced with adversarial training for a regularization term. This actively promotes the extraction of the coordination that fundamentally separates the two distributions. Third, best-of-N sampling is applied for action selection during rollouts in order to give directionality toward/beyond the regions where the coordination is insufficiently extracted/sufficiently discovered. The proposed method consistently improves the sample efficiency and the final control performance over the conventional diagonal distribution modeling across three locomotion tasks with high-dimensional action spaces. Furthermore, the analysis indicates that this improvement is achieved by reducing variance in certain action dimensions, effectively coupling them with other actions as action coordination.

## Loss-Guided Tsetlin Machines: Making the Learning Objective Explicit

DevFeed: [Loss-Guided Tsetlin Machines: Making the Learning Objective Explicit](<https://devfeed.tech/articles/loss-guided-tsetlin-machines-making-the-learning-objective-explicit-145145.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.26114>)

Author: Mayur Shende, Runar Helin, Ole-Christoffer Granmo, Vladimir I. Zadorozhny

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Model Development](<https://devfeed.tech/topics/model-development.md>), [Calibrated uncertainty](<https://devfeed.tech/topics/calibrated-uncertainty.md>)

Tags: [classification](<https://devfeed.tech/tags/classification.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [interpretability](<https://devfeed.tech/tags/interpretability.md>), [loss-functions](<https://devfeed.tech/tags/loss-functions.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>)

### AI overview

Loss-Guided Tsetlin Machines replace the fixed, implicit margin-based update rule with a swappable explicit loss that drives clause feedback and weight updates, while leaving the Tsetlin Automaton and clause logic unchanged. The paper evaluates cross entropy (CE), symmetric cross entropy (SCE), and asymmetric loss (ASL) on six multiclass, multilabel, and binary classification datasets. With a suitable loss, accuracy and F1 match or exceed the baseline on five datasets; ASL improves macro F1 by up to 9 percentage points on imbalanced multilabel data. CE reduces calibration error by a factor of 3 to 11 across all six datasets. A qualitative per-class inspection on image and text data reports that clause-based global interpretability is preserved.

### Source excerpt

arXiv:2611.26114v1 Announce Type: new Abstract: The Tsetlin Machine (TM) has no explicit notion of a loss function. Its learning signal is fixed to an implicit, margin-based update rule (MBUR), with no way to choose an objective function suited to a specific task. This paper introduces Loss-Guided TM, which replaces MBUR with an explicit and swappable loss function. In our proposed method, clause feedback and weight updates are driven by the loss, while keeping the Tsetlin Automaton and clause logic unchanged. The approach incorporates cross entropy (CE), symmetric cross entropy (SCE), and asymmetric loss (ASL). We evaluate Loss-Guided TM across six datasets that span multiclass, multilabel, and binary classification tasks. With a suitable loss, Loss-Guided TM matches or improves accuracy and F1 over MBUR on five of the six datasets. The gains are greatest for the imbalanced multilabel datasets, where ASL improves macro F1 by up to 9 percentage points. With CE, Loss-Guided TM is also significantly better calibrated than MBUR on all six datasets, reducing calibration error by a factor of 3 to 11. A qualitative inspection shows that the clause-based global interpretability, examined at the per-class level on image and text data, is preserved when using the Loss-Guided mechanism. These results show that our explicit TM learning objective allows direct targeting of uncertainty calibration, data imbalance, and task structure, opening the TM to the broader literature on loss functions.

## Why a Computer Scientist Thinks Superintelligence Won't Happen

DevFeed: [Why a Computer Scientist Thinks Superintelligence Won't Happen](<https://devfeed.tech/articles/why-a-computer-scientist-thinks-superintelligence-won-t-happen-145256.md>)

Original publisher: [Read original article](<https://spectrum.ieee.org/ai-superintelligence-debate>)

Author: Cal Newport

Published: 2026-11-19T12:00:03Z

Content type: opinion

Language: en

Sources: [IEEE Spectrum](<https://devfeed.tech/sources/ieee-spectrum.md>)

Topics: [superintelligence](<https://devfeed.tech/topics/superintelligence.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [ai alignment](<https://devfeed.tech/topics/ai-alignment.md>), [Agentic AI](<https://devfeed.tech/topics/what-is-agentic-ai.md>), [agentic workflows](<https://devfeed.tech/topics/agentic-workflows.md>), [computational complexity](<https://devfeed.tech/topics/computational-complexity.md>), [AI Engineering](<https://devfeed.tech/topics/ai-engineering.md>), [Model Development](<https://devfeed.tech/topics/model-development.md>), [AI Foundation Models](<https://devfeed.tech/topics/ai-foundation-models.md>)

Tags: [agi](<https://devfeed.tech/tags/agi.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-companies](<https://devfeed.tech/tags/ai-companies.md>), [alignment](<https://devfeed.tech/tags/alignment.md>), [anthropic](<https://devfeed.tech/tags/anthropic.md>), [anthropic-ceo-dario-amodei](<https://devfeed.tech/tags/anthropic-ceo-dario-amodei.md>), [artificial-general-intelligence](<https://devfeed.tech/tags/artificial-general-intelligence.md>), [asi](<https://devfeed.tech/tags/asi.md>), [autonomous](<https://devfeed.tech/tags/autonomous.md>), [capabilities](<https://devfeed.tech/tags/capabilities.md>), [computer](<https://devfeed.tech/tags/computer.md>), [control](<https://devfeed.tech/tags/control.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [engineers](<https://devfeed.tech/tags/engineers.md>), [extinction](<https://devfeed.tech/tags/extinction.md>), [large-language-models](<https://devfeed.tech/tags/large-language-models.md>), [openai](<https://devfeed.tech/tags/openai.md>), [recursive-self-improvement](<https://devfeed.tech/tags/recursive-self-improvement.md>), [rsi](<https://devfeed.tech/tags/rsi.md>), [science](<https://devfeed.tech/tags/science.md>), [singularity](<https://devfeed.tech/tags/singularity.md>), [superintelligence](<https://devfeed.tech/tags/superintelligence.md>)

### AI overview

The author argues that recursive self-improvement leading to superintelligence remains highly speculative: post-training depends on suitable feedback data, major AI advances require substantial scientific work, and compute is finite. Computer science also shows that some problems are unsolvable or practically intractable, while AI progress remains uneven across tasks. The article attributes reported hacking incidents to long-running agents that let LLM outputs guide actions with few constraints, and contrasts these with task-specific systems such as AlphaFold and autonomous driving. It warns that pursuing self-improving systems may still increase risks by making AI harder to understand and control, and calls for more independent scrutiny.

### Source excerpt

Many people were caught off guard this past summer when employees of one of the world's leading AI companies began casually debating whether their products would lead to human extinction. It started when Anthropic researcher Jacob Coxon announced his resignation, warning that his employer and OpenAI were "racing straight to self-improving superintelligence and gambling with our lives." Evan Hubinger, Anthropic's head of alignment science, replied, "Jacob is correct here--we really do earnestly believe AI could kill all humans!" And Anthropic CEO Dario Amodei subsequently told CNN, "I agree with Jacob much more than I disagree with him." These concerns stem from a belief that has circulated in futurist circles for many decades: Once engineers design an AI system capable of improving itself, it will trigger an accelerating cycle of autonomous upgrades known as recursive self-improvement (RSI). The result will be artificial superintelligence (ASI), a system with cognitive capabilities so vastly exceeding those of humans that it will easily evade our control and perhaps even wipe out our species. Artificial superintelligence is, it should be noted, a far more ambitious goal than artificial general intelligence (AGI), which generally refers to AI that can match human capabilities across a broad range of cognitive tasks. These ideas have prompted varied reactions. Some argue that the concept of ASI is being used for marketing or to instigate regulation that will help the incumbent labs squash new competitors. Others focus on the plausibility of such systems killing us all, debating the finer points of specific scenarios such as AI-engineered superviruses. Still others claim that aligning ASI to human interests is feasible and so we should rush ahead to usher in this digital utopia. Amid these conversations, one question has received comparatively little attention: From an engineering perspective, is superintelligence even possible? The answer is important because it can he

## Element-Wise Femoral Stress Prediction: A Hybrid Graph Neural Network Surrogate for Finite Element Analysis

DevFeed: [Element-Wise Femoral Stress Prediction: A Hybrid Graph Neural Network Surrogate for Finite Element Analysis](<https://devfeed.tech/articles/element-wise-femoral-stress-prediction-a-hybrid-graph-neural-network-surrogate-for-finite-element-analysis-143502.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.24274>)

Author: Sikha O K, Abdollah Moslemi, Elham Alizadeh, Ludovic Humbert, Miguel A. Gonzalez Ballester, Jerome Noailly

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Neural Network](<https://devfeed.tech/topics/neural-network.md>), [Predictive AI](<https://devfeed.tech/topics/predictive-ai.md>), [Model Development](<https://devfeed.tech/topics/model-development.md>)

Tags: [ablation](<https://devfeed.tech/tags/ablation.md>), [aggregation](<https://devfeed.tech/tags/aggregation.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [finite-element-analysis](<https://devfeed.tech/tags/finite-element-analysis.md>), [gnn](<https://devfeed.tech/tags/gnn.md>), [graph](<https://devfeed.tech/tags/graph.md>), [hybrid-graph](<https://devfeed.tech/tags/hybrid-graph.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [mesh](<https://devfeed.tech/tags/mesh.md>), [model](<https://devfeed.tech/tags/model.md>), [modeling](<https://devfeed.tech/tags/modeling.md>), [neural-network](<https://devfeed.tech/tags/neural-network.md>), [performance](<https://devfeed.tech/tags/performance.md>), [prediction](<https://devfeed.tech/tags/prediction.md>), [residual](<https://devfeed.tech/tags/residual.md>), [risk-assessment](<https://devfeed.tech/tags/risk-assessment.md>), [sage](<https://devfeed.tech/tags/sage.md>), [spatial](<https://devfeed.tech/tags/spatial.md>), [stress](<https://devfeed.tech/tags/stress.md>), [structure](<https://devfeed.tech/tags/structure.md>)

### AI overview

The study proposes a graph neural network surrogate for predicting stress across femoral finite element meshes, aiming to reduce the computational burden of finite element analysis while retaining mesh structure that conventional machine learning may miss. Its GCN-SAGE-Residual architecture combines graph convolution, neighborhood aggregation, and residual learning. In a comparison of 14 models, it achieved the strongest reported performance (R² 0.8022; mean absolute error 2.879 MPa); an architectural ablation examined the contribution of its components. The abstract presents the approach as a computationally efficient alternative for orthopedic analysis and fracture risk assessment, but provides no further validation details here.

### Source excerpt

arXiv:2611.24274v1 Announce Type: new Abstract: Hip fractures represent a significant clinical challenge because they cause excess disability and mortality, and have a growing incidence in aging populations, as bone fragility increases. Although finite element analysis (FEA) provides biomechanical assessment of femoral strength, its clinical applicability is mainly limited by high computational cost. Conventional machine learning (ML) approaches have been explored as surrogate models for FEA; predictions are fast but fail to capture the spatial and topological structure of finite element meshes. To address these limitations, this study proposes a hybrid Graph Neural Network (GNN)-based framework for femoral stress prediction using graph-structured finite element modeling (FEM) data. The proposed GCN-SAGE-Residual architecture combines graph convolution, neighborhood aggregation, and residual learning to model spatial dependencies and stress propagation across mesh elements. We systematically evaluated 14 models in a primary comparison, spanning classical ML methods and baseline GNN architectures, complemented by a targeted architectural ablation that isolates the contribution of each component of the proposed design. Evaluation results demonstrate that the proposed hybrid architecture achieves the strongest predictive performance among all evaluated models. The proposed hybrid model achieves the best performance with an R^2 score of 0.8022 and mean absolute error of 2.879 MPa, highlighting the importance of incorporating mesh topology in biomechanical prediction tasks. The proposed framework provides a scalable and computationally efficient alternative for FEM-based orthopedic analyses and fracture risk assessment. The code is publicly available on GitHub.

## MLE Agents: Autonomous ML Engineering in Large-scale Recommendation Systems

DevFeed: [MLE Agents: Autonomous ML Engineering in Large-scale Recommendation Systems](<https://devfeed.tech/articles/mle-agents-autonomous-ml-engineering-in-large-scale-recommendation-systems-142723.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.23910>)

Author: Matthew Li, Albert Liang, Dilip Raj

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [agentic workflows](<https://devfeed.tech/topics/agentic-workflows.md>), [recommendation systems](<https://devfeed.tech/topics/recommendation-systems.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Model Development](<https://devfeed.tech/topics/model-development.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>)

Tags: [agents](<https://devfeed.tech/tags/agents.md>), [autonomous](<https://devfeed.tech/tags/autonomous.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [industry](<https://devfeed.tech/tags/industry.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [recommendation](<https://devfeed.tech/tags/recommendation.md>), [recommendation-systems](<https://devfeed.tech/tags/recommendation-systems.md>)

### AI overview

Human-on-the-loop MLE agents proposed, implemented, trained, and evaluated changes to large-scale search rankers, while engineers retained control of objectives, search direction, and final review. To address experiments taking hours, offline metrics that may not predict online impact, and diminishing gains, the system used parallel training, an objective calibrated to online impact, and careful context engineering. In one campaign, 325 of 1,097 completed candidate experiments were accepted (29.6%), versus 15.6% in an earlier campaign with a different objective and search constraint. The best model increased search volume by 0.23% in a large-scale A/B test. The authors report that some gains came from previously untried architectural changes and emphasize that the harness around the model matters alongside the underlying model.

### Source excerpt

arXiv:2611.23910v1 Announce Type: new Abstract: The typical workflow for improving large-scale ranking and recommendation models is slow and tedious: an engineer forms a hypothesis, implements it as a code change, trains a model, and evaluates it both offline and online, often relying on their own intuition and experience along the way. Autonomous LLM agents promise to speed this up by running the loop on their own, but it is still unclear whether they can do this in industry-scale systems rather than on benchmarks or small proxy tasks. We build human-on-the-loop machine learning engineer (MLE) agents that propose, implement, train, and evaluate model changes on large-scale search rankers. Engineers retain control over objective design, search direction, and final-model review. We identify three obstacles that make this industry setting challenging: feedback takes hours per experiment, offline metrics do not always predict online impact, and improvements saturate after the early wins. We address these with parallel training runs, an objective calibrated to match online impact, and careful context engineering. In one campaign, the system ran 1,097 completed candidate experiments and accepted 325, a 29.6\% acceptance rate compared with 15.6\% in the earlier campaign under a different objective and search constraint. Its best model increases search volume by 0.23\% in an A/B test of large scale. Several improvements were architectural changes the team had never tried, drawn from the broader industry and literature, suggesting that MLE agents can explore regions of the design space that human intuition overlooks. We find that the design of the harness around the model matters: intuition suggests that a stronger underlying model is enough, but in our experience, the design of the loop around it is just as critical. More broadly, our work shows that autonomous agents can do real machine-learning engineering in industry-scale systems.

## Measuring What Matters: Synthetic Benchmarks for Concept Bottleneck Models

DevFeed: [Measuring What Matters: Synthetic Benchmarks for Concept Bottleneck Models](<https://devfeed.tech/articles/measuring-what-matters-synthetic-benchmarks-for-concept-bottleneck-models-142528.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2606.04326>)

Author: Julian Skirzynski, Harry Cheon, Shreyas Kadekodi, Meredith Stewart, Berk Ustun

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Ground truth / benchmark quality](<https://devfeed.tech/topics/ground-truth-benchmark-quality.md>), [Model Development](<https://devfeed.tech/topics/model-development.md>), [AI Models](<https://devfeed.tech/topics/ai-models.md>), [human feedback](<https://devfeed.tech/topics/human-feedback.md>)

Tags: [annotation](<https://devfeed.tech/tags/annotation.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [automation](<https://devfeed.tech/tags/automation.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [bottleneck](<https://devfeed.tech/tags/bottleneck.md>), [concepts](<https://devfeed.tech/tags/concepts.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [data](<https://devfeed.tech/tags/data.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [decision](<https://devfeed.tech/tags/decision.md>), [experts](<https://devfeed.tech/tags/experts.md>), [failure](<https://devfeed.tech/tags/failure.md>), [failures](<https://devfeed.tech/tags/failures.md>), [interpretability](<https://devfeed.tech/tags/interpretability.md>), [models](<https://devfeed.tech/tags/models.md>), [performance](<https://devfeed.tech/tags/performance.md>), [quality](<https://devfeed.tech/tags/quality.md>), [real-world](<https://devfeed.tech/tags/real-world.md>), [supervision](<https://devfeed.tech/tags/supervision.md>), [support](<https://devfeed.tech/tags/support.md>), [synthetic](<https://devfeed.tech/tags/synthetic.md>)

### AI overview

The paper introduces synthetic benchmarks for concept bottleneck models, addressing the shortage of real-world datasets with concept labels. Researchers can generate labeled data and vary modality, concept selection, and annotation quality to study which problems suit these models and isolate causes of failure. The authors demonstrate evaluation for decision support, where models help human experts, and automation of routine tasks without supervision. They report that the benchmarks reveal failure modes and help attribute them to model architecture or data.

### Source excerpt

arXiv:2606.04326v2 Announce Type: replace-cross Abstract: Concept bottleneck models predict using high-level concepts detected in inputs. Although these models provide a simple way to reap benefits from interpretability, they are hard to train and evaluate because very few real-world datasets include concept labels. In this paper, we develop synthetic benchmarks for concept bottleneck models. With our benchmarks, researchers can generate labeled datasets and control properties that affect performance, such as data modality, the choice of concepts, or annotation quality. They can thus find problems that suit these models and isolate the factors behind their failures. We demonstrate how our benchmarks can be used to evaluate major use cases for concept bottleneck models: decision support, i.e., helping human experts make better decisions; and automation, i.e., handling routine tasks without supervision. Our demonstrations show how the benchmarks can identify new failure modes and attribute them to either the architecture or the data.

## Don't Pay Attention, PLANT It: Pretraining Attention via Learning-to-Rank

DevFeed: [Don't Pay Attention, PLANT It: Pretraining Attention via Learning-to-Rank](<https://devfeed.tech/articles/don-t-pay-attention-plant-it-pretraining-attention-via-learning-to-rank-143215.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2410.23066>)

Author: Debjyoti Saha Roy, Byron C. Wallace, Javed A. Aslam

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [text-classification](<https://devfeed.tech/topics/text-classification.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Model Development](<https://devfeed.tech/topics/model-development.md>)

Tags: [ablation](<https://devfeed.tech/tags/ablation.md>), [attention](<https://devfeed.tech/tags/attention.md>), [classification](<https://devfeed.tech/tags/classification.md>), [coding](<https://devfeed.tech/tags/coding.md>), [comparisons](<https://devfeed.tech/tags/comparisons.md>), [content](<https://devfeed.tech/tags/content.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [deepseek](<https://devfeed.tech/tags/deepseek.md>), [deepseek-v3](<https://devfeed.tech/tags/deepseek-v3.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [learning-to-rank](<https://devfeed.tech/tags/learning-to-rank.md>), [llama-3-1](<https://devfeed.tech/tags/llama-3-1.md>), [llms](<https://devfeed.tech/tags/llms.md>), [medical](<https://devfeed.tech/tags/medical.md>), [mistral](<https://devfeed.tech/tags/mistral.md>), [plug](<https://devfeed.tech/tags/plug.md>), [pretraining](<https://devfeed.tech/tags/pretraining.md>), [recommendation](<https://devfeed.tech/tags/recommendation.md>), [text-classification](<https://devfeed.tech/tags/text-classification.md>), [topic](<https://devfeed.tech/tags/topic.md>), [training](<https://devfeed.tech/tags/training.md>)

### AI overview

PLANT initializes an extreme multi-label classifier's label-attention module by training a Learning-to-Rank model on mutual information gain, then fine-tunes end to end over an LLM's output representations. The plug-and-play approach is reported to outperform state-of-the-art methods on ICD coding, legal topic classification, and content recommendation, and to improve fine-tuned LLMs in controlled nonmedical comparisons. Gains are largest for rare labels, though a training-free LLM extraction baseline remains competitive on rarely seen labels. Ablations identify attention initialization as a key contributor. The supplied text is an abstract, so it does not provide detailed evaluation results.

### Source excerpt

arXiv:2410.23066v5 Announce Type: replace-cross Abstract: State-of-the-art Extreme Multi-Label Text Classification models rely on multi-label attention to focus on key tokens in input text, but learning good attention weights is challenging. We introduce PLANT (Pretrained and Leveraged AtteNTion), a plug-and-play strategy for initializing the label-attention module that operates on top of an LLM's output representations. PLANT works by planting label-specific attention using a Learning-to-Rank model trained on mutual information gain, before end-to-end fine-tuning. This architecture-agnostic approach integrates with LLM backbones (e.g., Mistral-7B, Llama-3.1-8B, DeepSeek-V3, and Phi-3-mini). PLANT outperforms SOTA methods across tasks like ICD coding, legal topic classification, and content recommendation, and improves fine-tuned LLMs in controlled comparisons outside the medical domain. Gains are largest on rare labels, although a training-free LLM extraction baseline remains competitive on rarely seen labels. Ablation studies confirm that attention initialization is a key driver of these gains. We make our code available at https://github.com/deb-official/plant

## Pragmatic Curiosity: A Unified Framework for Hybrid Learning and Optimization via Active Inference

DevFeed: [Pragmatic Curiosity: A Unified Framework for Hybrid Learning and Optimization via Active Inference](<https://devfeed.tech/articles/pragmatic-curiosity-a-unified-framework-for-hybrid-learning-and-optimization-via-active-inference-143231.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2602.06104>)

Author: Yingke Li, Anjali Parashar, Enlu Zhou, Chuchu Fan

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Model Development](<https://devfeed.tech/topics/model-development.md>), [Hyperparameter optimization](<https://devfeed.tech/topics/hyperparameter-optimization.md>)

Tags: [bayesian](<https://devfeed.tech/tags/bayesian.md>), [black-box](<https://devfeed.tech/tags/black-box.md>), [coverage](<https://devfeed.tech/tags/coverage.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [curiosity](<https://devfeed.tech/tags/curiosity.md>), [decision](<https://devfeed.tech/tags/decision.md>), [experimental-design](<https://devfeed.tech/tags/experimental-design.md>), [framework](<https://devfeed.tech/tags/framework.md>), [hybrid](<https://devfeed.tech/tags/hybrid.md>), [inference](<https://devfeed.tech/tags/inference.md>), [information](<https://devfeed.tech/tags/information.md>), [learning](<https://devfeed.tech/tags/learning.md>), [monitoring](<https://devfeed.tech/tags/monitoring.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [risk](<https://devfeed.tech/tags/risk.md>), [search](<https://devfeed.tech/tags/search.md>), [stat-ml](<https://devfeed.tech/tags/stat-ml.md>)

### AI overview

Pragmatic Curiosity (PraC) unifies Bayesian optimization and Bayesian experimental design for sequential workflows where evaluations are costly and learning is coupled to improving task performance. It scores candidate queries by balancing information gain about a task-relevant latent symbol against expected regret, with design choices around what to clarify, how to encode task value, and the balance between information and regret. The abstract describes three settings: monitoring with fixed global symbols and known losses; active search with induced local symbols and evolving coverage goals; and composite Bayesian optimization that learns hierarchical regret under unknown preferences. It reports reduced downstream decision risk, better coverage of critical outcomes, and joint learning of predictive and preference structures without task-specific staging rules.

### Source excerpt

arXiv:2602.06104v3 Announce Type: replace Abstract: Many engineering and scientific workflows rely on expensive black-box evaluations, requiring sequential decisions that must both improve task performance and reduce uncertainty. Bayesian optimization (BO) and Bayesian experimental design (BED) provide powerful but largely separate treatments of goal-directed optimization and information-seeking experimentation, leaving limited guidance for hybrid settings in which learning and optimization are intrinsically coupled. We propose Pragmatic Curiosity (PraC), a unified framework for hybrid learning and optimization via active inference. PraC evaluates candidate queries by trading information gain about a task-relevant latent symbol against an expected regret-based potential over outcomes. This formulation foregrounds three operational design choices: which latent quantity should be clarified, how task value is encoded as regret, and how strongly information gain should be exchanged against pragmatic regret. We instantiate PraC across three regimes of increasing complexity: decision-oriented monitoring with fixed global symbols and known downstream losses, targeted active search with induced local symbols and evolving coverage goals, and composite Bayesian optimization with hierarchical regret learning under unknown preferences. Across these regimes, PraC reduces downstream decision risk, improves coverage of critical outcome regions, and jointly learns predictive and preference structures without relying on task-specific staging rules.

## Enhancing Active Learning Performance with Initial Data Selection Based on Foundation Models

DevFeed: [Enhancing Active Learning Performance with Initial Data Selection Based on Foundation Models](<https://devfeed.tech/articles/enhancing-active-learning-performance-with-initial-data-selection-based-on-foundation-models-142789.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.24384>)

Author: Mohammad Mirzaee, Mehrshad Fallah Estalkhzir, Mohammad Reza Mohammadi

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Model Development](<https://devfeed.tech/topics/model-development.md>), [AI Foundation Models](<https://devfeed.tech/topics/ai-foundation-models.md>), [clustering](<https://devfeed.tech/topics/clustering.md>), [image recognition](<https://devfeed.tech/topics/image-recognition.md>)

Tags: [clustering](<https://devfeed.tech/tags/clustering.md>), [consistency](<https://devfeed.tech/tags/consistency.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [data](<https://devfeed.tech/tags/data.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [detection](<https://devfeed.tech/tags/detection.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [foundation-model](<https://devfeed.tech/tags/foundation-model.md>), [foundation-models](<https://devfeed.tech/tags/foundation-models.md>), [framework](<https://devfeed.tech/tags/framework.md>), [image](<https://devfeed.tech/tags/image.md>), [labeling](<https://devfeed.tech/tags/labeling.md>), [learning](<https://devfeed.tech/tags/learning.md>), [model-performance](<https://devfeed.tech/tags/model-performance.md>), [object-detection](<https://devfeed.tech/tags/object-detection.md>), [performance](<https://devfeed.tech/tags/performance.md>), [pool](<https://devfeed.tech/tags/pool.md>), [resource](<https://devfeed.tech/tags/resource.md>), [results](<https://devfeed.tech/tags/results.md>), [robustness](<https://devfeed.tech/tags/robustness.md>), [sampling](<https://devfeed.tech/tags/sampling.md>)

### AI overview

The framework uses YOLO-World feature clustering and objectness scores to select and refine the initial labeled pool for active object detection, instead of uncertainty sampling in the first cycle; later cycles use consistency-based sampling. Evaluated on Aquarium and NWPU VHR-10, it reports an average 6.6% increase in image-based mAP50 on NWPU, while image refinement improves object-based evaluation by approximately 11%. The abstract also reports gains in model performance and labeling time, but does not quantify the latter.

### Source excerpt

arXiv:2611.24384v1 Announce Type: cross Abstract: In active object detection, selecting the initial labeled pool is essential, especially in complex datasets with expensive labeling and low resource budget. This study presents an active learning framework designed to optimize the initial training cycle of active learning for the object detection task. Our approach diverges from traditional methods by utilizing feature-based clustering from the YOLO-World model and refining images based on objectness scores rather than relying on uncertainty sampling in the initial cycle. This approach focuses on finding samples with various features and emphasizes training in areas where the foundation model is most confident. Doing so helps the model better understand different types of objects. For subsequent cycles, we employ a consistency-based sampling technique to maintain the robustness of the training process. To evaluate our method, we use two complex datasets called Aquarium and NWPU VHR-10, which are well known for their diverse object sizes and challenging conditions. In the NWPU dataset, our method increases mAP50 by an average of 6.6% in image-based evaluation. Moreover, the image refinement method shows an improvement of approximately 11% in object-based evaluation. The results show improvements in model performance and labeling time, demonstrating the advantages of using foundation models in early cycles of active learning. An implementation of the algorithm is available at https://github.com/Mhrshd-fllh/ALBoost-FM

## Learning with Unlearning: Synaptic Role Partitioning for Simultaneous Continual Learning and Machine Unlearning

DevFeed: [Learning with Unlearning: Synaptic Role Partitioning for Simultaneous Continual Learning and Machine Unlearning](<https://devfeed.tech/articles/learning-with-unlearning-synaptic-role-partitioning-for-simultaneous-continual-learning-and-machine-unlearning-143493.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.24205>)

Author: Mohammad Ali Vahedifar, Behzad Bozorgtabar, Qi Zhang

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Neural Network](<https://devfeed.tech/topics/neural-network.md>), [Model Development](<https://devfeed.tech/topics/model-development.md>)

Tags: [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [capacity](<https://devfeed.tech/tags/capacity.md>), [continual-learning](<https://devfeed.tech/tags/continual-learning.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [distillation](<https://devfeed.tech/tags/distillation.md>), [framework](<https://devfeed.tech/tags/framework.md>), [gradient](<https://devfeed.tech/tags/gradient.md>), [image](<https://devfeed.tech/tags/image.md>), [image-generation](<https://devfeed.tech/tags/image-generation.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [machine-unlearning](<https://devfeed.tech/tags/machine-unlearning.md>), [partitioning](<https://devfeed.tech/tags/partitioning.md>)

### AI overview

LwU aims to combine continual learning with machine unlearning by assigning each weight a retention score and a deletion score, then dividing weights into four zones with separate update rules. It freezes weights needed only for retention, clears weights needed only for deletion and reuses them as capacity, and updates contested weights with a gradient-ascent step projected orthogonally to the retention gradient. Remaining capacity learns incoming tasks through teacher-guided self-distillation. The abstract says deletion is handled post hoc without retraining. It reports that LwU matches or outperforms the strongest baseline across CL and MU jobs, and outperforms the closest baseline in image generation and dangerous-content removal. It also reports a real-world demonstration on a Physical Artificial Intelligence dataset; the excerpt gives no further experimental details.

### Source excerpt

arXiv:2611.24205v1 Announce Type: new Abstract: Continual Learning (CL) asks a model to acquire new knowledge without forgetting what it already knows. Machine Unlearning (MU) asks it to destroy specific knowledge without damaging the rest. The challenge is integrating them in the same system; the two objectives pull on the same weights, and an edit made for one harms the other. To address this, we propose Learning with Unlearning (LwU), a unified framework that resolves the conflict where it occurs: at the individual synapse. LwU scores every weight twice, once against the data to keep and once against the data to delete, and partitions the network into four zones with separate update rules. Weights that only require retention are frozen. Weights that only require deletion are cleared and returned to the network as free capacity. Contested weights take a gradient-ascent step projected orthogonally to the retention gradient. The remaining capacity learns the incoming task by a privileged teacher via self-distillation. Deletion is served post hoc, and no retraining is required. Across both CL and MU jobs, LwU outperforms or matches the strongest baseline. Its real-world practicality has been demonstrated on the Physical Artificial Intelligence (PAI) dataset we conducted. Additionally, in the image generation task and in removing dangerous content, it outperformed the closest baseline, as shown in Fig.1.

## Counterfactual Feature Regularization for Expansion-Based Class-Incremental Learning

DevFeed: [Counterfactual Feature Regularization for Expansion-Based Class-Incremental Learning](<https://devfeed.tech/articles/counterfactual-feature-regularization-for-expansion-based-class-incremental-learning-142510.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2603.09145>)

Author: Zhen Zhang, Jielei Chu, Jiangtao Hu, Bin Liu, Jie Wang, Ya Liu, Fengmao Lv, Tianrui Li

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Model Development](<https://devfeed.tech/topics/model-development.md>), [Representation learning](<https://devfeed.tech/topics/representation-learning.md>)

Tags: [cause](<https://devfeed.tech/tags/cause.md>), [completeness](<https://devfeed.tech/tags/completeness.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [effectiveness](<https://devfeed.tech/tags/effectiveness.md>), [experimental](<https://devfeed.tech/tags/experimental.md>), [feature](<https://devfeed.tech/tags/feature.md>), [generator](<https://devfeed.tech/tags/generator.md>), [incremental](<https://devfeed.tech/tags/incremental.md>), [learning](<https://devfeed.tech/tags/learning.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [networks](<https://devfeed.tech/tags/networks.md>), [probability](<https://devfeed.tech/tags/probability.md>), [reliability](<https://devfeed.tech/tags/reliability.md>), [risk](<https://devfeed.tech/tags/risk.md>), [robust](<https://devfeed.tech/tags/robust.md>), [semantic](<https://devfeed.tech/tags/semantic.md>), [space](<https://devfeed.tech/tags/space.md>), [tasks](<https://devfeed.tech/tags/tasks.md>), [theoretical](<https://devfeed.tech/tags/theoretical.md>)

### AI overview

Expansion-based class-incremental learning can preserve old features yet still suffer collisions when new task features rely on shortcuts or confuse visually similar classes across tasks. The proposed CPNS regularization extends Probability of Necessity and Sufficiency to measure intra-task causal completeness and inter-task separability. A twin-network counterfactual generator creates intra-task counterfactual features to reduce intra-task PNS risk and inter-task interfering features to reduce inter-task PNS risk. The authors report theoretical analysis of reliability and experiments showing effectiveness, describing the regularizer as plug-and-play for expansion-based CIL. The supplied text gives no experiment details or numerical results.

### Source excerpt

arXiv:2603.09145v4 Announce Type: replace-cross Abstract: Current expansion-based methods for Class Incremental Learning (CIL) effectively mitigate catastrophic forgetting by freezing old features. However, such task-specific features learned from the new task may collide with the old features. From a causal perspective, spurious feature correlations are the main cause of this collision, manifesting in two scopes: (i) guided by empirical risk minimization (ERM), intra-task spurious correlations cause task-specific features to rely on shortcut features. These non-robust features are vulnerable to interference, inevitably drifting into the feature space of other tasks; (ii) inter-task spurious correlations induce semantic confusion between visually similar classes across tasks. To address this, we propose a Probability of Necessity and Sufficiency (PNS)-based regularization method to guide feature expansion in CIL. Specifically, we first extend the definition of PNS to expansion-based CIL, termed CPNS, which quantifies both the causal completeness of intra-task representations and the separability of inter-task representations. We then introduce a dual-scope counterfactual generator based on twin networks to ensure the measurement of CPNS, which simultaneously generates: (i) intra-task counterfactual features to minimize intra-task PNS risk and ensure causal completeness of task-specific features, and (ii) inter-task interfering features to minimize inter-task PNS risk, ensuring the separability of inter-task representations. Theoretical analyses confirm its reliability. The regularization is a plug-and-play method for expansion-based CIL to mitigate feature collision. Extensive experiments demonstrate the effectiveness of the proposed method.

## Meta-RL with Bayesian Linear Task Models

DevFeed: [Meta-RL with Bayesian Linear Task Models](<https://devfeed.tech/articles/meta-rl-with-bayesian-linear-task-models-142501.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2512.20974>)

Author: Jingyang You, Hanna Kurniawati

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>), [Representation learning](<https://devfeed.tech/topics/representation-learning.md>), [linear-regression](<https://devfeed.tech/topics/linear-regression.md>), [Model Development](<https://devfeed.tech/topics/model-development.md>)

Tags: [basis](<https://devfeed.tech/tags/basis.md>), [bayesian](<https://devfeed.tech/tags/bayesian.md>), [bayesian-inference](<https://devfeed.tech/tags/bayesian-inference.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-ro](<https://devfeed.tech/tags/cs-ro.md>), [inference](<https://devfeed.tech/tags/inference.md>), [kernel](<https://devfeed.tech/tags/kernel.md>), [learning](<https://devfeed.tech/tags/learning.md>), [linear](<https://devfeed.tech/tags/linear.md>), [manipulation](<https://devfeed.tech/tags/manipulation.md>), [meta](<https://devfeed.tech/tags/meta.md>), [models](<https://devfeed.tech/tags/models.md>), [mujoco](<https://devfeed.tech/tags/mujoco.md>), [performance](<https://devfeed.tech/tags/performance.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [reward](<https://devfeed.tech/tags/reward.md>), [rl](<https://devfeed.tech/tags/rl.md>), [tasks](<https://devfeed.tech/tags/tasks.md>), [zero-shot](<https://devfeed.tech/tags/zero-shot.md>)

### AI overview

GLiBRL is a meta-reinforcement-learning framework for adapting to unseen tasks that replaces variational inference with conjugate Bayesian updates. It combines generalised linear task models with learnable nonlinear basis functions to make exact sequential updates to task parameters and model noise, and computes a closed-form marginal likelihood. Its permutation-invariant updates can be used with both off-policy and on-policy algorithms. The abstract also reports an exact kernel identity linking distances between learned task representations to kernel discrepancies over task contexts. In comparisons with eight representative or recent methods, GLiBRL had the highest aggregate zero-shot test performance on the MuJoCo locomotion and MetaWorld manipulation benchmarks.

### Source excerpt

arXiv:2512.20974v5 Announce Type: replace-cross Abstract: Deep Bayesian reinforcement learning adapts to unseen tasks by inferring latent transition and reward models, but existing methods typically rely on variational posteriors and evidence lower bounds, introducing approximation error and unstable task representations. We introduce GLiBRL, a deep Bayesian RL framework that combines generalised linear task models with learnable non-linear basis functions. GLiBRL features conjugate Bayesian inference, yielding exact, sequential posterior updates over task parameters and model noise, together with a closed-form marginal likelihood that eliminates variational inference. The update is naturally permutation-invariant, allowing GLiBRL to integrate with both off- and on-policy algorithms. GLiBRL also learns task representation admitting an exact kernel identity, relating distances between task representations to kernel discrepancies over the task contexts. Compared against eight representative or recent meta reinforcement learning methods, GLiBRL achieves the highest aggregate zero-shot test performance on both the MuJoCo locomotion and MetaWorld manipulation benchmarks.

## From Perception to Simulation: Unveiling and Empowering Mental Spatial Dynamics in MLLMs

DevFeed: [From Perception to Simulation: Unveiling and Empowering Mental Spatial Dynamics in MLLMs](<https://devfeed.tech/articles/from-perception-to-simulation-unveiling-and-empowering-mental-spatial-dynamics-in-mllms-140047.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.22541>)

Author: Jiachun Li, Zhuoran Jin, Kang Liu, Jun Zhao

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [Simulation](<https://devfeed.tech/topics/simulation.md>), [LLM evaluation / benchmarking](<https://devfeed.tech/topics/llm-evaluation-benchmarking.md>), [Model Development](<https://devfeed.tech/topics/model-development.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>)

Tags: [benchmark](<https://devfeed.tech/tags/benchmark.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [framework](<https://devfeed.tech/tags/framework.md>), [improvement](<https://devfeed.tech/tags/improvement.md>), [intelligence](<https://devfeed.tech/tags/intelligence.md>), [large-language-models](<https://devfeed.tech/tags/large-language-models.md>), [learning](<https://devfeed.tech/tags/learning.md>), [multimodal](<https://devfeed.tech/tags/multimodal.md>), [performance](<https://devfeed.tech/tags/performance.md>), [reasoning](<https://devfeed.tech/tags/reasoning.md>), [robust](<https://devfeed.tech/tags/robust.md>), [simulation](<https://devfeed.tech/tags/simulation.md>), [spatial](<https://devfeed.tech/tags/spatial.md>), [spatial-reasoning](<https://devfeed.tech/tags/spatial-reasoning.md>)

### AI overview

MentalGrid evaluates whether multimodal language models can simulate changing spatial states without intermediate visual feedback. The authors report that existing models struggle with perceptual demands in simpler scenarios and multi-step reasoning as tasks become more complex. They train SpatialWalker with a three-stage curriculum using 16,000 specialized samples; it improves 33.8% on average over its base model, exceeds Gemini-3-Flash on spatial simulation, and retains a 4.9% average gain on out-of-domain benchmarks. These are results reported in the supplied abstract.

### Source excerpt

arXiv:2611.22541v1 Announce Type: cross Abstract: Spatial reasoning is a fundamental component of world understanding for Multimodal Large Language Models (MLLMs). While existing research has advanced the perception of spatial relations in static scenes, the domain of dynamic spatial simulation remains relatively unexplored. To bridge this gap, we introduce MentalGrid, a benchmark focused on spatial reasoning tasks that require models to simulate evolving spatial states without intermediate visual feedback. Extensive evaluation on our benchmark reveals a significant deficiency in the spatial simulation capabilities of existing models. Specifically, we find that performance is constrained by perceptual limitations in simple simulation scenarios and hindered by inadequate multi-step reasoning as complexity increases. To address these challenges, we propose a three-stage curriculum learning framework utilizing 16k specialized spatial reasoning samples. This approach yields SpatialWalker, which achieves a 33.8% average improvement over its base model, surpassing proprietary models like Gemini-3-Flash in spatial simulation. Furthermore, SpatialWalker demonstrates robust generalization, maintaining a 4.9% average gain on out-of-domain benchmarks, demonstrating the pivotal role of spatial simulation as a fundamental building block for broader spatial intelligence.

## Beyond Success Rates: Trainability and Extractability for Offline GCRL

DevFeed: [Beyond Success Rates: Trainability and Extractability for Offline GCRL](<https://devfeed.tech/articles/beyond-success-rates-trainability-and-extractability-for-offline-gcrl-141540.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2602.05459>)

Author: Jan Malte T\\"opperwien, Aditya Mohan, Marius Lindauer

Published: 2026-11-18T05:00:00Z

Content type: opinion

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Hyperparameter optimization](<https://devfeed.tech/topics/hyperparameter-optimization.md>), [Model Development](<https://devfeed.tech/topics/model-development.md>), [Algorithms](<https://devfeed.tech/topics/algorithms.md>)

Tags: [algorithm](<https://devfeed.tech/tags/algorithm.md>), [algorithms](<https://devfeed.tech/tags/algorithms.md>), [crl](<https://devfeed.tech/tags/crl.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cube](<https://devfeed.tech/tags/cube.md>), [diagnostics](<https://devfeed.tech/tags/diagnostics.md>), [goals](<https://devfeed.tech/tags/goals.md>), [it](<https://devfeed.tech/tags/it.md>), [learning](<https://devfeed.tech/tags/learning.md>), [offline](<https://devfeed.tech/tags/offline.md>), [policy](<https://devfeed.tech/tags/policy.md>), [random](<https://devfeed.tech/tags/random.md>), [range](<https://devfeed.tech/tags/range.md>), [rank](<https://devfeed.tech/tags/rank.md>), [regression](<https://devfeed.tech/tags/regression.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [results](<https://devfeed.tech/tags/results.md>), [settings](<https://devfeed.tech/tags/settings.md>), [signal](<https://devfeed.tech/tags/signal.md>), [study](<https://devfeed.tech/tags/study.md>)

### AI overview

Offline GCRL success rates alone conceal how sensitive methods are to value-learning and policy-extraction hyperparameters. Under a shared extractor, the study maps learning-rate and extraction settings for GCIQL, GCIVL, QRL, and CRL. On AntMaze-M, 59% of QRL and 66% of CRL configurations reach at least 90% of each method's best sampled success, versus 3% for GCIQL and GCIVL; on Cube and Scene, GCIQL and GCIVL perform better across broader hyperparameter regions. The authors examine advantage rankings of trajectory goals versus random goals and how strongly AWR focuses updates on a subset of transitions to interpret these differences. They recommend evaluating both attainable success and the hyperparameter range that sustains it, alongside post-hoc diagnostics of the policy-training signal.

### Source excerpt

arXiv:2602.05459v3 Announce Type: replace Abstract: Offline goal-conditioned reinforcement learning (GCRL) methods are typically benchmarked by the success rates of optimized hyperparameter configurations. However, success rate by itself does not provide information about the sensitivity of algorithms to hyperparameters for value learning or policy extraction, since an algorithm could obtain the same success rate across a wide range of hyperparameters, or might succeed only on specific settings. We study this phenomenon in the offline GCRL algorithms GCIQL, GCIVL, QRL, and CRL under a shared policy extractor, by constructing landscapes of interaction between the learning rate and policy extraction hyperparameters such as the temperature of advantage-weighted regression (AWR). These landscapes reveal large, task-dependent differences in hyperparameter sensitivity. On AntMaze-M, 59% of QRL configurations and 66% of CRL configurations achieve at least 90% of the method's best sampled success rate, compared with 3% for both GCIQL and GCIVL. On Cube and Scene, GCIQL and GCIVL instead attain higher success over broader hyperparameter regions than QRL and CRL. To interpret these differences, we analyze how reliably learned advantages rank actual goals from the same trajectory above random goals, and how strongly AWR concentrates policy updates on a small subset of transitions. Our findings indicate strongly differing behavior of these algorithms at policy-extraction time, even under the same success rates. Given these results, we argue that offline GCRL methods should be evaluated by both attainable success and the needed hyperparameter range to sustain it, complemented by post-hoc diagnostics to understand the signal used to train the policy. The overview and code are available at https://luh-ai.github.io/GCRL-Landscapes/ .

[Next page](<https://devfeed.tech/topics/model-development.md?cursor=WyIyMDI2LTExLTE4VDA1OjAwOjAwKzAwOjAwIiwgImQ0MmYyYjNjLTJkNGEtNDhhMy1hZDY3LTEzM2RmMjZmNjc4NSJd>)