# few-shot-learning

A machine learning problem setting in which a model generalizes to a new task or class from a very small number of labeled examples.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## Meta-LSQ: A Closed-Form Sobolev Inner Solver for Few-Shot Meta-Learning

DevFeed: [Meta-LSQ: A Closed-Form Sobolev Inner Solver for Few-Shot Meta-Learning](<https://devfeed.tech/articles/meta-lsq-a-closed-form-sobolev-inner-solver-for-few-shot-meta-learning-143547.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.24718>)

Author: Antonin Sulc

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [few-shot](<https://devfeed.tech/topics/few-shot.md>), [few-shot-learning](<https://devfeed.tech/topics/few-shot-learning.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>)

Tags: [basis](<https://devfeed.tech/tags/basis.md>), [beam](<https://devfeed.tech/tags/beam.md>), [cross-validation](<https://devfeed.tech/tags/cross-validation.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-lo](<https://devfeed.tech/tags/cs-lo.md>), [few-shot](<https://devfeed.tech/tags/few-shot.md>), [kernel](<https://devfeed.tech/tags/kernel.md>), [meta](<https://devfeed.tech/tags/meta.md>), [random](<https://devfeed.tech/tags/random.md>), [regression](<https://devfeed.tech/tags/regression.md>), [sensor](<https://devfeed.tech/tags/sensor.md>), [simulation](<https://devfeed.tech/tags/simulation.md>), [solve](<https://devfeed.tech/tags/solve.md>)

### AI overview

Meta-LSQ fits each task with one regularised least-squares solve over a fixed random Fourier basis, meta-learning only bandwidth, ridge and derivative weight. It can use derivative observations and yields exact closed-form derivatives. On 10-shot sinusoid regression, it trained in 14 seconds and achieved 5.5 times lower error than a first-order MAML baseline that trained 36 times longer; a Gaussian-process control suggests shared-kernel learning explains much of the gain. In simulated DESY beam tuning, it slightly outperformed an ALPaCA-style model when model derivatives were available and exceeded gradient descent. Across the full simulated target-beam range, its best setting had 7.3 μm median error versus 14.4 μm for the published final beams of a reinforcement-learning agent, though that agent had a lower median on 300 small-target trials. On real sensor-calibration data, Meta-LSQ beat MAML on two of four targets, while an exact Gaussian process with the same prior-mean change and an ALPaCA-style model were more accurate on most targets. It also broke down on non-smooth targets; for noiseless targets outside the basis span, generalised cross-validation selected too little ridge.

### Source excerpt

arXiv:2611.24718v1 Announce Type: new Abstract: Gradient-based meta-learners such as model-agnostic meta-learning (MAML) adapt to a new task with an inner loop of gradient descent that has to be differentiated through, and they meta-learn every network weight. Meta-LSQ instead fits each task with one regularised least-squares solve over a fixed random Fourier basis and meta-learns only three scalars, a bandwidth, a ridge and a derivative weight. Since the derivative of a sinusoid is again a sinusoid, the same solve can also take derivative observations, and the fitted model has exact closed-form derivatives of any order. On 10-shot sinusoid regression Meta-LSQ trains in 14 s and ends with a 5.5 times lower error than a first-order MAML baseline that trains 36 times longer, and a Gaussian-process control suggests that most of this gain comes from sharing one kernel across tasks. In simulated beam tuning at DESY's ARES facility with model derivatives it is slightly more accurate than an ALPaCA-style model and well ahead of gradient descent. Over DESY's full range of target beams, again in simulation and from measurements alone, its best setting has a median error of 7.3 $\mu$m, against 14.4 for the published final beams of their reinforcement-learning agent, which has the lower median on their 300 trials of small targets. On real sensor-calibration data a learned prior mean puts it ahead of MAML on two of four targets, but an exact Gaussian process with the same change and an ALPaCA-style model stay more accurate on most targets, and it breaks down on non-smooth targets. On noiseless targets outside the span of the basis, in-sample criteria such as generalised cross-validation choose too small a ridge.

## Weight-Space Programming Doesn't Need Transformers

DevFeed: [Weight-Space Programming Doesn't Need Transformers](<https://devfeed.tech/articles/weight-space-programming-doesn-t-need-transformers-139953.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.22055>)

Author: Hyunjong Ok, Jaeho Lee

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Frontier AI](<https://devfeed.tech/topics/frontier-ai.md>), [few-shot](<https://devfeed.tech/topics/few-shot.md>), [few-shot-learning](<https://devfeed.tech/topics/few-shot-learning.md>)

Tags: [architecture](<https://devfeed.tech/tags/architecture.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [gradient-descent](<https://devfeed.tech/tags/gradient-descent.md>), [large-language-model](<https://devfeed.tech/tags/large-language-model.md>), [memory](<https://devfeed.tech/tags/memory.md>), [table](<https://devfeed.tech/tags/table.md>), [transformer](<https://devfeed.tech/tags/transformer.md>), [weight-space-programming](<https://devfeed.tech/tags/weight-space-programming.md>)

### AI overview

In weight-space programming, an LLM writes model weights directly, so the authors test whether it should design a model rather than fill a fixed Transformer. With minimal guidance, ten labeled examples, and a sandbox, three frontier LLMs mainly solve multiple-choice tasks by encoding plain-text facts in a sparse table queried by a fixed procedure, reaching up to 54.4% test accuracy. One LLM-designed model beats the most accurate released Transformer written by that same LLM using about 1,800 times fewer parameters. The table makes weights interpretable and lets the LLM locate and rewrite errors; removing a row affects about one other answer. The authors propose an associative-memory model reported up to 17 times faster on GPUs than a same-sized Transformer. Pretrained LLMs remain more accurate, and the structural claim is specifically about models whose weights are written directly; the abstract also mentions tests on other benchmarks and answer-generating tasks without reporting their results.

### Source excerpt

arXiv:2611.22055v1 Announce Type: cross Abstract: Standard neural architectures such as the Transformer were chosen because they train well with gradient descent and run fast on GPUs. In weight-space programming, a large language model (LLM) writes target weights by hand without gradient-based training, so those reasons no longer apply. Should we keep the Transformer? Given minimal guidance, ten labeled examples, and a sandbox, mainly on multiple-choice questions, we task three frontier LLMs with either generating the weights of a fixed architecture or designing the architecture themselves. When free to design, all three compile plain-text facts into a sparse table queried by a fixed procedure, reaching up to 54.4% test accuracy. An LLM-designed model achieves higher test accuracy than the most accurate released Transformer that the same LLM wrote while using about 1,800x fewer parameters. Without gradients, they choose a table, in which the LLM can state what each weight means and can find and rewrite a wrong one. Each wrong answer points to one row, and removing a row changes about one other answer. From this, we propose an associative-memory-based model up to 17x faster on GPUs than a same-sized Transformer. We also test pretrained weights, other benchmarks, and tasks where models generate answers. Pretrained LLMs remain more accurate; the claim concerns which structure to build when writing weights.

## Network Intrusion Detection Adaptability under Spurious Labels: A Noise-Aware Few-Shot Class-Incremental Learning Framework

DevFeed: [Network Intrusion Detection Adaptability under Spurious Labels: A Noise-Aware Few-Shot Class-Incremental Learning Framework](<https://devfeed.tech/articles/network-intrusion-detection-adaptability-under-spurious-labels-a-noise-aware-few-shot-class-incremental-learning-framework-137037.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.21473>)

Author: Zijian Zhao, Davide Di Monda, Domenico Ciuonzo, Yipeng Wang, Antonio Pescap\\\`e

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>), [arXiv: Networking](<https://devfeed.tech/sources/arxiv-networking.md>)

Topics: [few-shot-learning](<https://devfeed.tech/topics/few-shot-learning.md>), [few-shot](<https://devfeed.tech/topics/few-shot.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [data augmentation](<https://devfeed.tech/topics/data-augmentation.md>), [Embeddings](<https://devfeed.tech/topics/embeddings.md>)

Tags: [classifiers](<https://devfeed.tech/tags/classifiers.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-ni](<https://devfeed.tech/tags/cs-ni.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [detection](<https://devfeed.tech/tags/detection.md>), [edge](<https://devfeed.tech/tags/edge.md>), [embeddings](<https://devfeed.tech/tags/embeddings.md>), [few-shot](<https://devfeed.tech/tags/few-shot.md>), [framework](<https://devfeed.tech/tags/framework.md>), [iiot](<https://devfeed.tech/tags/iiot.md>), [incremental](<https://devfeed.tech/tags/incremental.md>), [intrusion-detection](<https://devfeed.tech/tags/intrusion-detection.md>), [iot](<https://devfeed.tech/tags/iot.md>), [logistic-regression](<https://devfeed.tech/tags/logistic-regression.md>), [memory](<https://devfeed.tech/tags/memory.md>), [nearest-neighbor](<https://devfeed.tech/tags/nearest-neighbor.md>), [network](<https://devfeed.tech/tags/network.md>), [network-traffic](<https://devfeed.tech/tags/network-traffic.md>), [noise](<https://devfeed.tech/tags/noise.md>), [pretraining](<https://devfeed.tech/tags/pretraining.md>), [sampling](<https://devfeed.tech/tags/sampling.md>), [statistics](<https://devfeed.tech/tags/statistics.md>)

### AI overview

NAFN aims to make few-shot class-incremental network intrusion detection more resilient to mislabeled examples in newly observed attack classes. It detects and removes suspected mislabeled biflows, then estimates class-conditional embeddings from the remaining clean data, calibrates them with base-class statistics, and resamples synthetic features to restore the original biflow count. The framework uses the Rethink Few-Shot teacher-student paradigm with nearest-neighbor or logistic regression classifiers and different pretraining and memory-selection strategies. On CSE-CIC-IDS2018, Edge-IIoT, and IoT-NID, the abstract reports retained prior knowledge and strong incremental performance with up to 50% label noise; new-class F1 improved over non-denoised baselines by as much as 52%. It also reports time efficiency and robustness across networks and high-noise settings.

### Source excerpt

arXiv:2611.21473v1 Announce Type: new Abstract: Network Intrusion Detection Systems (NIDSs) must incrementally adapt to emerging attacks, even when only a limited amount of labeled network traffic is available. Consequently, Few-Shot Class-Incremental Learning (FSCIL)-based NIDSs have gained growing attention. However, existing FSCIL-based NIDSs are highly vulnerable to label noise in newly-observed attack classes. To address this limitation, we propose NAFN, a noise-aware FSCIL-based NIDS framework that reduces the impact of mislabeled data during incremental adaptation. NAFN consists of two components: (i) Noise Detection, which identifies and removes mislabeled biflows using noise-detection algorithms, and (ii) Feature Sampling, which estimates class-conditional embeddings from clean biflows, calibrates them with base-class statistics, and resamples synthetic features to recover the original biflow count. The framework follows the Rethink Few-Shot (RFS) teacher-student paradigm with lightweight nearest-neighbor or logistic regression classifiers, and various pretraining & memory selection strategies. Experiments on CSE-CIC-IDS2018, Edge-IIoT, and IoT-NID datasets show that NAFN effectively preserves prior knowledge and maintains strong incremental performance even under 50% label noise, consistently improving new-class F1-scores over non-denoised baselines, up to +52%, while being time-efficient and robust in cross-network and high-noise incremental settings. The implementation is publicly available to foster reproducibility.

## Vision-Language Model-Based Phase-OTDR Event Classification

DevFeed: [Vision-Language Model-Based Phase-OTDR Event Classification](<https://devfeed.tech/articles/vision-language-model-based-phase-otdr-event-classification-139100.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.20841>)

Author: Ahmet Bestami Kose, Mehmet Sergen Catal, Muhammet Cagri Yeke, Kivilcim Yuksel, Abdurrahman Gumus

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [vlm](<https://devfeed.tech/topics/vlm.md>), [few-shot](<https://devfeed.tech/topics/few-shot.md>), [few-shot-learning](<https://devfeed.tech/topics/few-shot-learning.md>), [image recognition](<https://devfeed.tech/topics/image-recognition.md>), [fourier transform](<https://devfeed.tech/topics/fourier-transform.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [adapter](<https://devfeed.tech/tags/adapter.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [based](<https://devfeed.tech/tags/based.md>), [but](<https://devfeed.tech/tags/but.md>), [cache](<https://devfeed.tech/tags/cache.md>), [class](<https://devfeed.tech/tags/class.md>), [classes](<https://devfeed.tech/tags/classes.md>), [classification](<https://devfeed.tech/tags/classification.md>), [classifier](<https://devfeed.tech/tags/classifier.md>), [clip](<https://devfeed.tech/tags/clip.md>), [code](<https://devfeed.tech/tags/code.md>), [color](<https://devfeed.tech/tags/color.md>), [configuration](<https://devfeed.tech/tags/configuration.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [event](<https://devfeed.tech/tags/event.md>), [language](<https://devfeed.tech/tags/language.md>), [language-model](<https://devfeed.tech/tags/language-model.md>), [model](<https://devfeed.tech/tags/model.md>), [vision](<https://devfeed.tech/tags/vision.md>), [vision-language-model](<https://devfeed.tech/tags/vision-language-model.md>)

### AI overview

The study finds that few-shot adaptation of CLIP visual features can classify Phase-OTDR events with limited labels. It converts raw samples into RGB images of STFT dB-magnitude spectrogram mosaics, then adapts Tip-Adapter-F cache keys while leaving both encoders unchanged. In a leakage-controlled evaluation with 20 support-sampling trials and a fixed held-out test set, ViT-L-14 with OpenAI weights and 128 shots per class achieved 96.81 ± 0.42% accuracy and 96.79 ± 0.42% macro-F1. With 16 shots per class (96 support images across six classes), accuracy was 90.17 ± 0.81%. Tip-Adapter-F outperformed zero-shot CLIP and training-free Tip-Adapter; reported analyses found performance stable across Vision Transformer backbones and prompt wording at the selected setting. The source says code and the image dataset are publicly available.

### Source excerpt

arXiv:2611.20841v1 Announce Type: cross Abstract: Phase-sensitive optical time-domain reflectometry (Phase-OTDR) enables distributed vibration sensing, but reliable event classification has traditionally relied on fully supervised models that require extensive annotated data. This study examines the adaptability of a Vision-Language Model (VLM) for Phase-OTDR event recognition through few-shot learning. Raw samples are converted into Short-Time Fourier Transform (STFT) dB-magnitude spectrogram mosaics, color-mapped and resized to RGB images, and processed by a Contrastive Language-Image Pre-training (CLIP) image encoder; few-shot adaptation optimizes only the Tip-Adapter-F cache keys, leaving both encoders unchanged. Under a leakage-controlled protocol (train-side support/validation pool of 512 images per class, fixed held-out test of 120 per class, no test-set access during selection, 20 distinct support-sampling trials), the selected configuration (the ViT-L-14 Vision Transformer encoder with OpenAI weights, 128 shots per class) reaches 96.81 +/- 0.42% accuracy and a macro-averaged F1 score of 96.79 +/- 0.42% on the held-out test set; a label-efficient 16-shot setting, that is, 96 labeled support images at adaptation time across the six classes, already attains 90.17 +/- 0.81%. Tip-Adapter-F clearly outperforms both zero-shot CLIP and the standard training-free Tip-Adapter, and prompt and backbone analyses show that performance is stable across Vision Transformer encoders and insensitive to prompt wording at the selected operating point. Overall, CLIP visual features carry a transferable structure that a lightweight support cache turns into a label-efficient Phase-OTDR event classifier. The code and the corresponding image dataset are made publicly available on GitHub to support further research: https://github.com/miralab-ai/vlm-phase-otdr.

## Geometric Stability and Sample Complexity for Compositional Function Priors

DevFeed: [Geometric Stability and Sample Complexity for Compositional Function Priors](<https://devfeed.tech/articles/geometric-stability-and-sample-complexity-for-compositional-function-priors-138883.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.19523>)

Author: Christopher L. Hammond

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [few-shot-learning](<https://devfeed.tech/topics/few-shot-learning.md>), [linear-regression](<https://devfeed.tech/topics/linear-regression.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [also](<https://devfeed.tech/tags/also.md>), [ambient](<https://devfeed.tech/tags/ambient.md>), [approximation](<https://devfeed.tech/tags/approximation.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [atlas](<https://devfeed.tech/tags/atlas.md>), [base](<https://devfeed.tech/tags/base.md>), [budget](<https://devfeed.tech/tags/budget.md>), [class](<https://devfeed.tech/tags/class.md>), [complexity](<https://devfeed.tech/tags/complexity.md>), [connect](<https://devfeed.tech/tags/connect.md>), [covering](<https://devfeed.tech/tags/covering.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [curved](<https://devfeed.tech/tags/curved.md>), [d](<https://devfeed.tech/tags/d.md>), [data](<https://devfeed.tech/tags/data.md>), [delta](<https://devfeed.tech/tags/delta.md>), [dimension](<https://devfeed.tech/tags/dimension.md>), [dimension-reduction](<https://devfeed.tech/tags/dimension-reduction.md>), [function](<https://devfeed.tech/tags/function.md>), [sample](<https://devfeed.tech/tags/sample.md>), [stat-ml](<https://devfeed.tech/tags/stat-ml.md>)

### AI overview

The correction size ε links data curvature to statistical complexity in compositional learning. For predictors f(x)=g(Ax+Δh(x)), covering-number bounds interpolate between an intrinsic regime--parametric in the kd entries of A and nonparametric in representation dimension k--and an ambient d-dimensional nonparametric rate, with the interpolation controlled by ε. When data lie near a k-dimensional manifold and the target depends only on the nearest manifold point, local models factorize using a tangent chart; their correction's Sobolev norm scales with curvature. Patching these models yields a finite-sample excess-risk bound whose ambient term vanishes with curvature. A one-dimensional converse shows affine compositional predictors incur an error floor on curved data unless their correction budget scales with curvature; the matching curved class attains the intrinsic rate.

### Source excerpt

arXiv:2611.19523v1 Announce Type: cross Abstract: We study when compositional structure yields dimension reduction in supervised learning. We frame learning as empirical risk minimization with a function-space prior: a regularization functional selects a subset of functions with favorable approximation and metric entropy. The models are compositions $f(x)=g(Ax+\Delta h(x))$, where $A\in\mathbb{R}^{k\times d}$ is a linear base map, $\Delta h$ is a nonlinear correction, and $g$ is an outer function; that is, $f$ is an arbitrary smooth function of a $k$-dimensional representation of the input that is linear up to a correction. We prove covering-number bounds showing that the statistical complexity of this class interpolates between an intrinsic regime, parametric in the $kd$ entries of the linear map and nonparametric only in the representation dimension $k$, and an ambient $d$-dimensional nonparametric rate; the interpolation is controlled by the correction size $\varepsilon=\|\Delta h\|$. We then connect $\varepsilon$ to data geometry. If the data lie in a small tube around a $k$-dimensional manifold and the regression target depends on the input only through its nearest point on that manifold, then locally the target factorizes exactly as $g(Ax+\Delta h(x))$ with $A$ a tangent chart and a correction whose Sobolev norm scales linearly with curvature at the localization scale. Patching such local models over an atlas gives a finite-sample excess-risk bound for a least-squares estimator over a finite net of local models, whose ambient term vanishes with curvature. For a one-dimensional model problem we also prove a converse: on curved data, every affine compositional predictor faces an error floor unless its correction budget scales with curvature, while the matching curved model class attains the intrinsic rate. The correction size $\varepsilon$ is thus the effective-dimension parameter linking curvature to statistical complexity.

## Zero-Shot Weight-Space Programming for Prediction and Classification of Target Variables in Structured Datasets

DevFeed: [Zero-Shot Weight-Space Programming for Prediction and Classification of Target Variables in Structured Datasets](<https://devfeed.tech/articles/zero-shot-weight-space-programming-for-prediction-and-classification-of-target-variables-in-structured-datasets-137426.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.19292>)

Author: Eduardo C. Garrido-Merch\\'an

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [mlp](<https://devfeed.tech/topics/mlp.md>), [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [few-shot-learning](<https://devfeed.tech/topics/few-shot-learning.md>)

Tags: [classification](<https://devfeed.tech/tags/classification.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [gradient-descent](<https://devfeed.tech/tags/gradient-descent.md>), [hidden](<https://devfeed.tech/tags/hidden.md>), [inference](<https://devfeed.tech/tags/inference.md>), [large-language-model](<https://devfeed.tech/tags/large-language-model.md>), [linear](<https://devfeed.tech/tags/linear.md>), [network](<https://devfeed.tech/tags/network.md>), [prediction](<https://devfeed.tech/tags/prediction.md>), [problems](<https://devfeed.tech/tags/problems.md>), [regression](<https://devfeed.tech/tags/regression.md>), [rotation](<https://devfeed.tech/tags/rotation.md>), [synthetic](<https://devfeed.tech/tags/synthetic.md>), [weight-space-programming](<https://devfeed.tech/tags/weight-space-programming.md>), [zero-shot](<https://devfeed.tech/tags/zero-shot.md>)

### AI overview

The study tests whether an LLM can infer a supervised dataset's target and encode it directly as every weight and bias in a feed-forward network, without optimization, code execution, or a chance to evaluate the network first. Across toy, rotated synthetic, benchmark, and engineering regression problems, its frozen networks match the closed-form optimum on linear tasks and reach the generating process's irreducible error on three nonlinear tasks; on two of those they outperform fitted baselines, including the same architecture trained by gradient descent. It fails on two tasks where the target structure is oblique to the analyzed coordinates. Additional constructors reveal distinct failures: some produce no parameters on the hardest tasks, while another always responds but can perform worse than predicting the mean. A separate constructor solves one oblique-hinge task, and linear-separator training error predicts when a hidden layer helps better than the tested composite complexity index.

### Source excerpt

arXiv:2611.19292v1 Announce Type: new Abstract: Weight-space programming fixes every parameter of a feed-forward network by hand, with no gradient descent anywhere in the construction. Every demonstration so far starts from a target function handed to the model as a rule system, making it a compilation problem. A supervised dataset withholds that specification, so the procedure must do the statistical inference itself and then express the result in the parameterisation it was given. We ask whether a large language model can do both. It is shown a small dataset in full, with the architecture of a feed-forward network, and writes out every weight and bias as a literal number, with no optimizer, no gradient, no code execution, and no chance to evaluate it first. Each construction is sealed and pre-registers its expected score. The graded suite runs from two toy problems, through nine synthetic problems disguised by a hidden rotation, to the classical benchmarks and one real engineering regression. The frozen network matches the closed-form optimum wherever that optimum is linear, reaches the irreducible error of the generating process on three rungs where it is not, and beats every fitted baseline on two of those, including the same architecture trained by gradient descent. It fails on two rungs, both times because the structure to be found is oblique to the coordinates it analysed. Three further constructors separate two failure modes a single score conflates: one returns no parameter vector at all on the hardest rungs, another delivers everywhere while writing networks worse than predicting the mean. One of those failures proves avoidable: another constructor solves the oblique-hinge rung by a margin that survives multiple-comparison correction. What predicts whether the hidden layer earns its keep is the training error of a linear separator, which the composite complexity index misses.

## Interactive Program Induction with Language Model Priors

DevFeed: [Interactive Program Induction with Language Model Priors](<https://devfeed.tech/articles/interactive-program-induction-with-language-model-priors-135205.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.18485>)

Author: Robin Shing Moon Chan, Ryan Cotterell

Published: 2026-11-16T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [few-shot](<https://devfeed.tech/topics/few-shot.md>), [few-shot-learning](<https://devfeed.tech/topics/few-shot-learning.md>), [Interactive AI Systems](<https://devfeed.tech/topics/interactive-ai-systems.md>)

Tags: [22](<https://devfeed.tech/tags/22.md>), [algorithm](<https://devfeed.tech/tags/algorithm.md>), [bayesian](<https://devfeed.tech/tags/bayesian.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [fewer](<https://devfeed.tech/tags/fewer.md>), [induction](<https://devfeed.tech/tags/induction.md>), [interactive](<https://devfeed.tech/tags/interactive.md>), [language-model](<https://devfeed.tech/tags/language-model.md>), [learner](<https://devfeed.tech/tags/learner.md>), [oracle](<https://devfeed.tech/tags/oracle.md>), [program](<https://devfeed.tech/tags/program.md>), [sampling](<https://devfeed.tech/tags/sampling.md>), [user](<https://devfeed.tech/tags/user.md>)

### AI overview

The paper treats program induction as Bayesian inference: a language model supplies a prior over candidate programs, and a greedy learner queries an oracle for outputs on inputs chosen to be maximally informative. Because finite examples can fit many programs, the method uses a PAC-style relaxation of equivalence and importance sampling to estimate query value. The authors state that it terminates after O((1/γ) log(1/p⁰(f⋆))) informative queries and recovers the target up to approximate equivalence when no other class outweighs it under the prior. On CodeARC, it improves accuracy over an LM-only agent by up to 22 percentage points while using up to 10.5 fewer input-output examples.

### Source excerpt

arXiv:2611.18485v1 Announce Type: cross Abstract: Program induction refers to the process in which a learner aims to recover a program from a few given input-output examples. This paradigm broadens programming access by letting users demonstrate the rule-based behavior they want to apply to unseen data, instead of having to implement it themselves. However, recovering a program from finite input-output examples is fundamentally ambiguous, as a finite set of input-output examples is theoretically consistent with infinitely many programs whose output diverges on unseen inputs. We view this problem from a Bayesian perspective, where, given a prior $p^0$ over programs, identifying the target program means concentrating posterior probability mass on it by observing additional input-output examples. In the interactive program induction setting, the learner chooses maximally informative inputs and queries an oracle, such as the user, for their outputs. We derive a greedy query-selection algorithm and show that it terminates after $\mathcal{O}(\frac{1}{\gamma}\log 1/p^0(f^\star))$ informative oracle queries, and returns the target program $f^\star$ up to approximate equivalence when no other class outweighs it under the prior. The algorithm uses a language model as the prior under a PAC-style relaxation of program equivalence, where the query-selection objective is estimated using importance sampling. On the CodeARC benchmark, the algorithm improves accuracy over the LM only agent by up to 22 percentage points while observing up to 10.5 fewer input-output examples.

## GenPFN: An In-Context Tabular Foundation Model Approach for Augmenting Small Datasets in Engineering

DevFeed: [GenPFN: An In-Context Tabular Foundation Model Approach for Augmenting Small Datasets in Engineering](<https://devfeed.tech/articles/genpfn-an-in-context-tabular-foundation-model-approach-for-augmenting-small-datasets-in-engineering-132052.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.14804>)

Author: Noah J. Bagazinski, Cyril Picard, Faez Ahmed

Published: 2026-11-13T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [data augmentation](<https://devfeed.tech/topics/data-augmentation.md>), [AI Foundation Models](<https://devfeed.tech/topics/ai-foundation-models.md>), [few-shot-learning](<https://devfeed.tech/topics/few-shot-learning.md>)

Tags: [1-4](<https://devfeed.tech/tags/1-4.md>), [abstract](<https://devfeed.tech/tags/abstract.md>), [adversarial](<https://devfeed.tech/tags/adversarial.md>), [approach](<https://devfeed.tech/tags/approach.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [autoencoder](<https://devfeed.tech/tags/autoencoder.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [candidate](<https://devfeed.tech/tags/candidate.md>), [case](<https://devfeed.tech/tags/case.md>), [case-study](<https://devfeed.tech/tags/case-study.md>), [class](<https://devfeed.tech/tags/class.md>), [comparisons](<https://devfeed.tech/tags/comparisons.md>), [context](<https://devfeed.tech/tags/context.md>), [coverage](<https://devfeed.tech/tags/coverage.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [foundation](<https://devfeed.tech/tags/foundation.md>), [foundation-model](<https://devfeed.tech/tags/foundation-model.md>), [model](<https://devfeed.tech/tags/model.md>), [small](<https://devfeed.tech/tags/small.md>)

### AI overview

GenPFN generates and evaluates engineering design candidates to augment small datasets for regression training. It uses a fixed-weight pretrained TabPFN classifier to guide interpolation between desired designs, with gated classifier gradients and Gaussian noise; candidates are then checked against problem constraints and objectives, and evaluated desired designs are added to the training context. Across fourteen selected problems, paired comparisons report higher class satisfaction and lower realism distance than the tested adversarial, VAE, and diffusion generators, while the VAE has greater coverage and diversity. Compared with SMOTE, GenPFN has lower coverage distance and greater diversity. In observed paired fits, augmentation reduced estimated AutoGluon validation MSE by 62.1% versus context-only training. In a hull-stern case study, 99.3% of generated candidates were propeller-compatible, compared with 1.4% of randomly sampled context designs.

### Source excerpt

arXiv:2611.14804v1 Announce Type: new Abstract: Costly simulations and experiments limit the data available for engineering design. This paper proposes GenPFN, a training-free candidate generator guided by a pretrained tabular Prior-Data Fitted Network (TabPFN) classifier with fixed weights. The context contains evaluated designs labeled desired (Class 1) or undesired (Class 0). GenPFN interpolates between desired designs and refines candidates using gated classifier gradients and Gaussian noise. Each candidate is evaluated using the problem's constraints and objective functions. Evaluated desired designs and their objective values augment the context for regression training. Evaluation spans fourteen selected problems, including a two-parameter benchmark and a 44-parameter hull problem. In paired comparisons, GenPFN with TabPFN v1 achieves higher class satisfaction and lower realism distance than the tested adversarial, variational-autoencoder and diffusion generators. The variational-autoencoder generator provides greater coverage and diversity. Relative to the synthetic minority over-sampling technique (SMOTE), GenPFN provides lower coverage distance and greater diversity. Among observed paired regression fits, augmentation reduces estimated AutoGluon validation mean squared error by 62.1% relative to context-only training. A hull-stern case study yields 99.3% propeller-compatible candidates versus 1.4% among randomly sampled context designs. Generation followed by evaluation produces sample sets that augment limited datasets and support training more accurate predictive models.

## Scikit-Learn Matched or Beat Jev's Calibration on BANKING77

DevFeed: [Scikit-Learn Matched or Beat Jev's Calibration on BANKING77](<https://devfeed.tech/articles/scikit-learn-matched-or-beat-jev-s-calibration-on-banking77-127371.md>)

Original publisher: [Read original article](<https://towardsdatascience.com/scikit-learn-matched-or-beat-jevs-calibration-on-banking77/>)

Author: Ankur Tomar

Published: 2026-11-10T16:30:00Z

Content type: comparison

Language: en

Sources: [Towards Data Science](<https://devfeed.tech/sources/towards-data-science.md>)

Topics: [scikit-learn](<https://devfeed.tech/topics/scikit-learn.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [few-shot](<https://devfeed.tech/topics/few-shot.md>), [few-shot-learning](<https://devfeed.tech/topics/few-shot-learning.md>), [Open Source Models & Datasets](<https://devfeed.tech/topics/open-source-models-datasets.md>), [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>)

Tags: [api](<https://devfeed.tech/tags/api.md>), [banking](<https://devfeed.tech/tags/banking.md>), [cascade](<https://devfeed.tech/tags/cascade.md>), [choice](<https://devfeed.tech/tags/choice.md>), [data](<https://devfeed.tech/tags/data.md>), [data-science](<https://devfeed.tech/tags/data-science.md>), [editor-s-picks](<https://devfeed.tech/tags/editor-s-picks.md>), [jev](<https://devfeed.tech/tags/jev.md>), [llm](<https://devfeed.tech/tags/llm.md>), [llm-applications](<https://devfeed.tech/tags/llm-applications.md>), [model-calibration](<https://devfeed.tech/tags/model-calibration.md>), [scikit-learn](<https://devfeed.tech/tags/scikit-learn.md>)

### AI overview

On BANKING77, temperature-scaled logistic regression matched Jev's Choice calibration on a seven-intent slice (ECE 0.022 vs. 0.014; the paired comparison was inconclusive) and had lower ECE on all 77 intents (0.031 vs. Jev's 0.065). The open model required 10,003 labeled queries; Jev required no training and used up to three examples per intent. Both systems still auto-acted on out-of-schema queries because a forced-choice schema lacks a "none of these" path, and separately framed Choice/Noul questions could disagree at a shared threshold. Threshold performance varied sharply by backend, and even calibrated scores missed the target precision by 2.6 points in one test; the author recommends choosing thresholds with labeled data and providing an abstention route. Jev's Choice answers were well calibrated on the small slice, while its yes/no answers were underconfident on true cases. Findings are limited to one short-text banking dataset, small Jev samples, and test-set-selected cascade operating points; Jev's cascade behavior was not tested.

### Source excerpt

Jev needs no training and at most three examples per intent; the open rebuild needs labels. Both still fall into the none-of-the-above trap. The post Scikit-Learn Matched or Beat Jev's Calibration on BANKING77 appeared first on Towards Data Science.

## Scout Before You Route: Predicting Task Success and Token Count from a Small LLM Prefill

DevFeed: [Scout Before You Route: Predicting Task Success and Token Count from a Small LLM Prefill](<https://devfeed.tech/articles/scout-before-you-route-predicting-task-success-and-token-count-from-a-small-llm-prefill-123343.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.09028>)

Author: Aristides Milios, Christopher Pal

Published: 2026-11-09T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Model Routing](<https://devfeed.tech/topics/model-routing.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [few-shot-learning](<https://devfeed.tech/topics/few-shot-learning.md>), [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>)

Tags: [cost](<https://devfeed.tech/tags/cost.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [llm](<https://devfeed.tech/tags/llm.md>), [model](<https://devfeed.tech/tags/model.md>), [prefill](<https://devfeed.tech/tags/prefill.md>), [reasoning](<https://devfeed.tech/tags/reasoning.md>), [route](<https://devfeed.tech/tags/route.md>), [token](<https://devfeed.tech/tags/token.md>)

### AI overview

A small frozen LLM can use prefill activations to predict candidate models' success probabilities and output lengths, enabling per-query cost estimates for routing. With reasoning models, perfect length knowledge could cut spending 29-61% on eight of nine datasets; the learned predictions save 9-33% at matched accuracy versus median-length pricing, reaching up to 69% of the ideal savings. Length is largely problem-dependent: on most math and code datasets, harder problems tend to produce longer outputs, so success predictions alone approximate length-based pricing; this breaks down on knowledge and mixed-format data. A length probe retains most savings with 20 labelled examples and transfers without labels to four of seven new benchmarks. Savings plateau around a 2B scout, while scouts above 4B cost more than the additional savings justify. These results are reported for the evaluated datasets and models.

### Source excerpt

arXiv:2611.09028v1 Announce Type: cross Abstract: Routers typically choose a model per query by trading predicted correctness against cost. Most price each model at a fixed cost, as if every query used the same number of output tokens, but the output length of reasoning models varies by an order of magnitude across queries. When is per-query cost worth predicting? What factors drive cost? How cheaply can predictions be made? We predict both the success probability and the output length of candidate models from the prefill activations of a single small frozen LLM, the scout. With reasoning candidates, perfect per-query cost knowledge would save 29-61% of spending on eight of nine datasets, and much less with non-reasoning candidates on the same problems. On those eight datasets, our predicted costs save 9-33% at matched accuracy relative to median-length pricing, up to 69% of what perfect knowledge would save. They are significantly cheaper than each estimator we reimplemented or adapted from published work, including ZeroRouter and a MixLLM-style predictor, on a majority of datasets. Output length depends mostly on the problem. On most math and code datasets, harder problems are longer, so predicted success alone prices cost nearly as well. On knowledge and mixed-format datasets they are not, and predicted success falls well behind a probe trained on length. The probe keeps most of its saving with 20 labelled problems and transfers without labels to four of seven new benchmarks. On a mixture of all nine benchmarks, it saves 22% beyond pricing each benchmark at its own median length. Savings stop growing at about a 2B scout, so above 4B a larger scout costs more than it adds.

## TraceViT: Grounded Trace Supervision for Visual Abstract Reasoning

DevFeed: [TraceViT: Grounded Trace Supervision for Visual Abstract Reasoning](<https://devfeed.tech/articles/tracevit-grounded-trace-supervision-for-visual-abstract-reasoning-120066.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2607.29586>)

Author: Binnan Liu, Yechi Ma, Tian Xie, Wei Hua

Published: 2026-11-06T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [few-shot](<https://devfeed.tech/topics/few-shot.md>), [few-shot-learning](<https://devfeed.tech/topics/few-shot-learning.md>), [generalization in machine learning](<https://devfeed.tech/topics/generalization-in-machine-learning.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [abstraction](<https://devfeed.tech/tags/abstraction.md>), [alignment](<https://devfeed.tech/tags/alignment.md>), [arc](<https://devfeed.tech/tags/arc.md>), [code](<https://devfeed.tech/tags/code.md>), [computer-vision](<https://devfeed.tech/tags/computer-vision.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [data](<https://devfeed.tech/tags/data.md>), [few-shot](<https://devfeed.tech/tags/few-shot.md>), [grid](<https://devfeed.tech/tags/grid.md>), [grounding](<https://devfeed.tech/tags/grounding.md>), [intermediate](<https://devfeed.tech/tags/intermediate.md>), [iteration](<https://devfeed.tech/tags/iteration.md>), [model](<https://devfeed.tech/tags/model.md>), [object](<https://devfeed.tech/tags/object.md>), [reasoning](<https://devfeed.tech/tags/reasoning.md>), [supervision](<https://devfeed.tech/tags/supervision.md>), [trace](<https://devfeed.tech/tags/trace.md>), [training](<https://devfeed.tech/tags/training.md>), [transformation](<https://devfeed.tech/tags/transformation.md>)

### AI overview

TraceViT trains a looped visual reasoner to refine ARC grid solutions through intermediate states, addressing the fact that conventional training constrains only the final output. It derives semantically monotonic transformation chains by rewriting and verifying programmatic solutions, then grounds each iteration in a task reference from the demonstrations and a workspace for the current grid. Soft trace alignment preserves the ordering of these states while allowing the number of iterations to vary. TraceViT reports 67.8% pass@2 on ARC-AGI-1 and 24.3% on ARC-AGI-2; ARC-AGI-1 ablations find trace supervision beneficial only when paired with grounding.

### Source excerpt

arXiv:2607.29586v2 Announce Type: replace-cross Abstract: The Abstraction and Reasoning Corpus (ARC) tests whether a model can infer an unseen transformation from a few input-output examples and apply it to a new grid. Looped visual reasoners refine predictions over multiple iterations, but conventional training constrains only the final output, leaving intermediate refinements unconstrained. We propose that these refinements should instead follow the transformation step by step. We introduce TraceViT, a looped visual reasoner trained with semantically monotonic transformation chains. We obtain these chains by rewriting and verifying programmatic task implementations, decomposing each solution into intermediate grid states. Each iteration is grounded by a task reference derived from the few-shot demonstrations and an object workspace representing the current grid state. Because these chains may differ in length from the loop, soft trace alignment enforces only their ordering, letting the model allocate iterations freely. TraceViT achieves 67.8% pass@2 on ARC-AGI-1 and 24.3% on ARC-AGI-2. Controlled ablations on ARC-AGI-1 show that trace supervision becomes beneficial only when paired with grounding. Code and data will be available at https://github.com/LiuBinnan/TraceViT.

## Molecular Prior Networks

DevFeed: [Molecular Prior Networks](<https://devfeed.tech/articles/molecular-prior-networks-117015.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.05103>)

Author: Pawel Gomoluch, Anthony Bradley

Published: 2026-11-04T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [few-shot](<https://devfeed.tech/topics/few-shot.md>), [few-shot-learning](<https://devfeed.tech/topics/few-shot-learning.md>), [Transformers](<https://devfeed.tech/topics/transformers.md>), [AI Foundation Models](<https://devfeed.tech/topics/ai-foundation-models.md>)

Tags: [affinity](<https://devfeed.tech/tags/affinity.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-ce](<https://devfeed.tech/tags/cs-ce.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [time](<https://devfeed.tech/tags/time.md>), [training](<https://devfeed.tech/tags/training.md>), [transformer](<https://devfeed.tech/tags/transformer.md>)

### AI overview

Molecular Prior Networks predicts query-molecule affinity labels from labeled support molecules in one transformer forward pass. Trained across thousands of existing affinity datasets, it aims to capture activity patterns across targets, chemical series, and dataset sizes without task-specific training. On the smallest support sets in the regression variant of FS-Mol, it matches state-of-the-art performance and runs faster than existing methods. Comparison with a reduced model equivalent to kernel regression in a meta-learned feature space suggests molecular similarity alone only partly explains its predictive performance.

### Source excerpt

arXiv:2611.05103v1 Announce Type: new Abstract: Predicting affinity between small molecules and their protein targets remains one of the central problems in drug discovery. Due to the substantial cost of wet-lab experiments, machine learning models of drug-target affinity are usually trained on relatively small datasets, ranging from tens to hundreds of molecules. Given the inherent difficulty of generalization between different targets and chemical series, existing datasets of protein-ligand affinities are typically not used for new projects. At the same time, tabular foundation models, including Prior-data Fitted Networks, have recently shown how transformers can be used to solve machine learning problems in a single forward pass of the network, after being trained over a large corpus of, typically, synthetic datasets. Inspired by that line of work, we propose a novel affinity modelling approach, in which the labels of the query molecules are predicted from the labelled support molecules in a single forward pass of a transformer model. The model is trained on thousands of existing affinity datasets, enabling it to capture the general patterns of molecular activity across different targets, chemical series and dataset sizes. For the smallest support sets, our model matches the state of the art on the regression variant of the FS-Mol benchmark, while being faster than existing methods, thanks to the lack of task-specific training. A comparison against a reduced variant of our architecture, which amounts to kernel regression in a meta-learned feature space, shows that the predictive performance of the transformer model is only partially explained by simple molecular similarity.

## Beyond Augmentation: Score-Guided Pathological Prior for EEG-based Depression Detection

DevFeed: [Beyond Augmentation: Score-Guided Pathological Prior for EEG-based Depression Detection](<https://devfeed.tech/articles/beyond-augmentation-score-guided-pathological-prior-for-eeg-based-depression-detection-116713.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2606.00180>)

Author: Xiaojing Chen, Jingqi Cheng, Xu Zhao, Wan Jiang, Jingjing Wu

Published: 2026-11-04T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [few-shot](<https://devfeed.tech/topics/few-shot.md>), [few-shot-learning](<https://devfeed.tech/topics/few-shot-learning.md>), [data augmentation](<https://devfeed.tech/topics/data-augmentation.md>)

Tags: [architecture](<https://devfeed.tech/tags/architecture.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [data-augmentation](<https://devfeed.tech/tags/data-augmentation.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [deep](<https://devfeed.tech/tags/deep.md>), [deep-learning](<https://devfeed.tech/tags/deep-learning.md>), [depression](<https://devfeed.tech/tags/depression.md>), [detection](<https://devfeed.tech/tags/detection.md>), [diffusion](<https://devfeed.tech/tags/diffusion.md>), [eeg](<https://devfeed.tech/tags/eeg.md>), [few-shot](<https://devfeed.tech/tags/few-shot.md>), [generative](<https://devfeed.tech/tags/generative.md>), [hardware](<https://devfeed.tech/tags/hardware.md>), [high-density](<https://devfeed.tech/tags/high-density.md>), [model](<https://devfeed.tech/tags/model.md>), [models](<https://devfeed.tech/tags/models.md>), [module](<https://devfeed.tech/tags/module.md>), [network](<https://devfeed.tech/tags/network.md>), [quantity](<https://devfeed.tech/tags/quantity.md>), [sample](<https://devfeed.tech/tags/sample.md>), [score](<https://devfeed.tech/tags/score.md>)

### AI overview

The proposed Score-Guided Classification (SGC) framework detects major depressive disorder from EEG without synthesizing training samples. It uses an unsupervised generative network to estimate structural and statistical anomalies, robustly normalizes those scores, and fuses them with deep features to guide classification. A Cross-Channel Spatial Adaptation module maps across differing channel configurations in multi-center data. The abstract reports experiments on Mumtaz2016 and high-density MODMA, claiming effectiveness and generalizability without data augmentation or sample synthesis; it provides no quantitative results in the supplied text.

### Source excerpt

arXiv:2606.00180v3 Announce Type: replace Abstract: Deep learning-based Major Depressive Disorder (MDD) detection using Electroencephalography (EEG) is fundamentally constrained by the "small-sample dilemma." Prevailing generative data augmentation methods not only incur heavy computational overhead but also risk introducing synthetic noise, thereby blurring classification boundaries. To challenge the traditional "data quantity first" convention, we propose a novel framework "Beyond Augmentation": Score-Guided Classification (SGC). SGC does not synthesize pseudo-samples; instead, it utilizes an unsupervised generative network architecture to model the structural and statistical anomaly degrees of samples, serving as the core "Pathological Prior". This prior, after robust normalization, is explicitly fused with deep feature representations, thereby precisely guiding the classifier's decision boundary. Furthermore, to dynamically adapt to varying channel configurations, we propose a Cross-Channel Spatial Adaptation module, utilizing a spatial mapping mechanism to effectively resolve the hardware heterogeneity of mismatched channels in multi-center datasets. Extensive experiments on the Mumtaz2016 and high-density MODMA datasets demonstrate the effectiveness and exceptional generalizability of our method under the challenging "zero data augmentation" setting and at "zero sample synthesis cost". Keywords: Electroencephalography (EEG), Depression Detection, Anomaly Score, Diffusion Models, Few-Shot Learning

## A Circuit for In-Context Learning

DevFeed: [A Circuit for In-Context Learning](<https://devfeed.tech/articles/a-circuit-for-in-context-learning-117058.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.05344>)

Author: Andrew Gao, William Gao

Published: 2026-11-04T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [few-shot](<https://devfeed.tech/topics/few-shot.md>), [few-shot-learning](<https://devfeed.tech/topics/few-shot-learning.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Transformers](<https://devfeed.tech/topics/transformers.md>)

Tags: [attention](<https://devfeed.tech/tags/attention.md>), [circuit](<https://devfeed.tech/tags/circuit.md>), [classification](<https://devfeed.tech/tags/classification.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [few-shot](<https://devfeed.tech/tags/few-shot.md>), [gate](<https://devfeed.tech/tags/gate.md>), [language-model](<https://devfeed.tech/tags/language-model.md>), [learning](<https://devfeed.tech/tags/learning.md>), [lookup](<https://devfeed.tech/tags/lookup.md>), [memory](<https://devfeed.tech/tags/memory.md>), [patching](<https://devfeed.tech/tags/patching.md>)

### AI overview

In few-shot sentence classification with arbitrary labels, the studied transformers perform in-context learning through a gated associative lookup during a single forward pass. Middle-layer attention heads write each demonstration's class into label-token positions; upper-layer heads match the query's class to those stored keys and retrieve the corresponding labels. An early-layer signal gates this readout when a label has repeated, while separate heads suppress the most recently seen label. The authors report causal evidence from activation patching, ablations and steering, and say the computation recurs across three 3-4B models, a model trained from scratch, and one model family from 0.6B to 8B, surviving instruction tuning. The circuit also transfers to semantic concepts and meaningful labels. These findings concern the settings and models described in the abstract.

### Source excerpt

arXiv:2611.05344v1 Announce Type: cross Abstract: In-context learning (ICL) lets a pretrained language model perform a task specified by demonstrations in its prompt, with no parameter updates. With arbitrary labels the model must acquire a mapping it cannot have memorized, so the learning happens within a single forward pass. Prior work found candidate parts but not how they compose. We give the composition for few-shot sentence classification with arbitrary labels in three 3-4B-parameter transformers. The model implements a key-value memory over label-token positions: each label token's residual stream encodes the label's identity, and middle-layer attention heads write into it the class of the preceding demonstration as its key. At the query, upper-layer heads match the query's class against the stored keys and copy out the matching labels. An early-layer signal encoding whether any label has repeated gates this readout; in its absence the readers attend to the start of the prompt and no label is favored. A separate set of heads suppresses the most recently seen label. Each step is established causally by activation patching, head ablations against random-head baselines, and steering with the model's own signals. A softmax model of the readout with two constants per head, fit on one prompt distribution, predicts another. The computation is conserved across the three models, the named heads alone reproduce the behaviour, and a transformer trained from scratch grows the same parts. The circuit transfers to semantic concepts and to labels with meaning, and the same parts, with a causal gate, appear from 0.6B to 8B in one family and survive instruction tuning. ICL in this setting is a gated associative lookup built from a few identifiable component types, traced causally from demonstrations to prediction.

## A Novel Lightweight Multi-Scale Branching Encoder for Few-Shot Image Classification

DevFeed: [A Novel Lightweight Multi-Scale Branching Encoder for Few-Shot Image Classification](<https://devfeed.tech/articles/a-novel-lightweight-multi-scale-branching-encoder-for-few-shot-image-classification-114981.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.00099>)

Author: Oriod Malo (AI Lab UMT), Alexandre Xavier Falc\\~ao (IC), Laurent Najman (KUSTAR, LIGM)

Published: 2026-11-03T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [few-shot](<https://devfeed.tech/topics/few-shot.md>), [few-shot-learning](<https://devfeed.tech/topics/few-shot-learning.md>), [image recognition](<https://devfeed.tech/topics/image-recognition.md>), [Medical imaging](<https://devfeed.tech/topics/medical-imaging.md>)

Tags: [ablation](<https://devfeed.tech/tags/ablation.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [branch](<https://devfeed.tech/tags/branch.md>), [classification](<https://devfeed.tech/tags/classification.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [embedding](<https://devfeed.tech/tags/embedding.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [feature](<https://devfeed.tech/tags/feature.md>), [few-shot](<https://devfeed.tech/tags/few-shot.md>), [image](<https://devfeed.tech/tags/image.md>), [image-classification](<https://devfeed.tech/tags/image-classification.md>), [imaging](<https://devfeed.tech/tags/imaging.md>), [lightweight](<https://devfeed.tech/tags/lightweight.md>), [medical](<https://devfeed.tech/tags/medical.md>), [performance](<https://devfeed.tech/tags/performance.md>)

### AI overview

Sum12 is a lightweight convolutional encoder for Prototypical Networks that uses seven parallel branches with kernel-size combinations summing to 12 to capture local and broader spatial features. On Omniglot, CIFAR-FS, FC100, and miniImageNet, it generally improved low-shot classification over the basic ProtoNet encoder while keeping model size compact; comparisons also included Inception-inspired variants. Ablations examined fusion, dilation, branch composition, and receptive-field budget. Tests on MedMNIST 2D assessed supervised medical image classification. The abstract does not provide numerical results, and full end-to-end validation on the target embedded platform remains future work, including resolving observed implementation discrepancies.

### Source excerpt

arXiv:2611.00099v1 Announce Type: cross Abstract: Few-Shot Learning (FSL) addresses classification scenarios in which only a small number of labeled examples are available for each class. This paper presents the Sum12 Multi-Scale branching encoder, a lightweight convolutional architecture designed to improve the embedding stage of Prototypical Networks without substantially increasing computational cost. The encoder uses seven parallel convolutional branches whose kernel-size combinations sum to 12, allowing local and broader spatial structures to be represented within a unified Multi-Scale feature space. The proposed architecture was evaluated on Omniglot, CIFAR- FS, FC100, and miniImageNet under controlled few-shot learning protocols and compared with the original Prototypical Network encoder and Inception-inspired branch variants. Additional ablation studies have investigated fusion strategies, dilated convolutions, branch compositions, and receptive field budget. Beyond few-shot learning, the encoder was also tested on the MedMNIST 2D benchmark to assess its behavior in conventional supervised medical image classification. Across these experiments, Sum12 generally improved the low-shot classification performance with respect to the basic ProtoNet encoder while preserving a compact model size. These properties suggest that the proposed encoder may be suitable for low-data visual recognition and resource-constrained intelligent imaging applications. However, complete end-to- end validation on the target embedded platform, including resolution of the observed implementation discrepancies, remains future work.

## Diagnosing Prototype Estimation in Few-Shot Class-Incremental Learning: Training-Free Geometry Selection and Unlabeled-Query Prototype Rectification

DevFeed: [Diagnosing Prototype Estimation in Few-Shot Class-Incremental Learning: Training-Free Geometry Selection and Unlabeled-Query Prototype Rectification](<https://devfeed.tech/articles/diagnosing-prototype-estimation-in-few-shot-class-incremental-learning-training-free-geometry-selection-and-unlabeled-query-prototype-rectification-114326.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.00022>)

Author: Md Tanvir Hossain Saikat, Md Akib Haider, Md. Tahmid Islam Tomal, Md Hasanul Kabir

Published: 2026-11-03T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [few-shot](<https://devfeed.tech/topics/few-shot.md>), [few-shot-learning](<https://devfeed.tech/topics/few-shot-learning.md>)

Tags: [aircraft](<https://devfeed.tech/tags/aircraft.md>), [clip](<https://devfeed.tech/tags/clip.md>), [covariance](<https://devfeed.tech/tags/covariance.md>), [covariance-shrinkage](<https://devfeed.tech/tags/covariance-shrinkage.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [dataset](<https://devfeed.tech/tags/dataset.md>), [diagnosis](<https://devfeed.tech/tags/diagnosis.md>), [diagnostics](<https://devfeed.tech/tags/diagnostics.md>), [extension](<https://devfeed.tech/tags/extension.md>), [fallback](<https://devfeed.tech/tags/fallback.md>), [family](<https://devfeed.tech/tags/family.md>), [feature](<https://devfeed.tech/tags/feature.md>), [features](<https://devfeed.tech/tags/features.md>), [few-shot](<https://devfeed.tech/tags/few-shot.md>), [gap](<https://devfeed.tech/tags/gap.md>), [incremental](<https://devfeed.tech/tags/incremental.md>), [learning](<https://devfeed.tech/tags/learning.md>), [prototype](<https://devfeed.tech/tags/prototype.md>), [query](<https://devfeed.tech/tags/query.md>), [selection](<https://devfeed.tech/tags/selection.md>), [training](<https://devfeed.tech/tags/training.md>)

### AI overview

Across 16 strong encoder-dataset pairs in few-shot class-incremental learning, replacing five-shot novel-class means with complete-data means explains most of the measured gap before linear-probe refitting, supporting noisy prototype estimation as a key constraint. ShrinkSel selects prototype radius and covariance shrinkage using support data, with shared LDA as an exact fallback; it beats LDA on 15 pairs by 0.04-1.52 points and selects LDA on Aircraft. Query Prototype Rectification (QPR) uses gated unlabeled queries to adjust novel-class directions while preserving the selected geometry. On CLIP ViT-B/16 it adds 1.21-2.03 points, including with query-wise self-exclusion, but incurs a measurable base-class cost; its gains weaken with more supports or very small query cohorts. A weaker base-only encoder did not meet the same diagnostic criterion.

### Source excerpt

arXiv:2611.00022v1 Announce Type: cross Abstract: Frozen visual encoders make prototype heads competitive in few-shot class-incremental learning (FSCIL), but it remains unclear how much of the residual head gap is sensitive to noisy novel-class means rather than to the subsequent changes introduced by classifier refitting. On identical features, we construct a retrospective matched counterfactual: first replace each five-shot novel mean with its complete-data counterpart under fixed cosine scoring, then fit a linear probe. Across sixteen strong encoder dataset pairs, mean replacement accounts for the majority of this ordered gap. Shot and classwise diagnostics support a finite-sample mechanism, whereas a weaker base-only encoder does not satisfy the same criterion. This diagnosis motivates ShrinkSel, a support-only selector over prototype radius and covariance shrinkage whose family contains shared LDA as an exact fallback. Across sixteen evaluated pairs, it improves LDA on fifteen by 0.04-1.52 points and selects the exact LDA endpoint on Aircraft. We further introduce Query Prototype Rectification (QPR), a batch-transductive extension that uses gated unlabeled queries to update novel directions while preserving the selected geometry. On CLIP ViT-B/16, QPR adds 1.21-2.03 points; exact query-wise self-exclusion retains the benefit, showing that the gains do not require self-inclusion. The update repairs scarce novel prototypes at a measurable base-class cost and weakens with plentiful supports or very small cohorts. Thus, the evidence available for prototype estimation is a central constraint in frozen-feature FSCIL.

## Crop Mapping Evaluation under Sampling Bias

DevFeed: [Crop Mapping Evaluation under Sampling Bias](<https://devfeed.tech/articles/crop-mapping-evaluation-under-sampling-bias-114450.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.00794>)

Author: Akram Zaytar, Kemunto Ochwang'i, Caleb Robinson, Girmaw A. Tadesse, Gilles Q. Hacheme, Rahul Dodhia, Inbal Becker-Reshef, Juan M. Lavista Ferres

Published: 2026-11-03T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [few-shot-learning](<https://devfeed.tech/topics/few-shot-learning.md>)

Tags: [bias](<https://devfeed.tech/tags/bias.md>), [crop](<https://devfeed.tech/tags/crop.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [inference](<https://devfeed.tech/tags/inference.md>), [mapping](<https://devfeed.tech/tags/mapping.md>), [performance](<https://devfeed.tech/tags/performance.md>), [sampling](<https://devfeed.tech/tags/sampling.md>), [validation](<https://devfeed.tech/tags/validation.md>)

### AI overview

Crop-map validation on field labels can overstate performance across a mapped region when field teams preferentially visit accessible parcels. The paper estimates region-wide performance with weighted prediction-powered inference: predictions on unlabeled regional data provide an initial estimate, then labeled-point errors correct it, with greater weight on under-sampled conditions. In five European regions, simulated collection practices created deployment gaps up to 0.20 macro-F1. Under the most biased rule in each region, 5 km block cross-validation overestimated macro-F1 by up to 0.18; weighted PPI reduced the largest error across regions to 0.07. Importance-weighted cross-validation using the same weights performed nearly as well, indicating that the weights account for most of the improvement. On a Kenyan roadside dataset, estimated macro-F1 dropped from 0.93 under random cross-validation to 0.61, but this correction relied on a small number of heavily weighted labels. The Kenyan datasets lack region-wide reference labels, limiting direct verification there.

### Source excerpt

arXiv:2611.00794v1 Announce Type: new Abstract: Government agencies use crop maps to estimate production, monitor food security, and set agricultural policy. These crop maps are usually validated against field labels, but these labels are rarely a random sample of the area the map covers. Field teams tend to label parcels near roads and skip ones that are small or hard to reach, so performance measured on held-out labels mainly describes the places they visited. We call the difference between this reported performance and performance over the whole mapped region the deployment performance gap. To estimate performance over the whole region, we use a weighted form of prediction-powered inference (PPI). The classifier's predictions on unlabeled points across the region give a first estimate, and its errors on the labeled points correct that estimate, with more weight on labels from under-sampled conditions. We test this in five European regions where crop registers label every declared parcel, so the true performance is known. Simulating six documented collection practices there produces gaps of up to 0.20 macro-F1. On the collection rule that creates the largest gap in each region, 5 km block cross-validation still overestimates macro-F1 by as much as 0.18, while PPI with weights estimated from satellite embeddings reduces the largest error across regions to 0.07. Importance-weighted cross-validation with the same weights does almost as well, so most of the improvement comes from the weights themselves. We also apply the method to three Kenyan datasets that have no region-wide reference labels. For a roadside dataset collected with helmet-mounted cameras, the estimated macro-F1 falls from 0.93 under random cross-validation to 0.61, although the correction relies on a small number of heavily weighted labels. Our code is available at https://github.com/microsoft/pappi.

## Episodic looking - using reinforcement learning to bias perception enhances few-shot and continual learning with protection from forgetting

DevFeed: [Episodic looking - using reinforcement learning to bias perception enhances few-shot and continual learning with protection from forgetting](<https://devfeed.tech/articles/episodic-looking-using-reinforcement-learning-to-bias-perception-enhances-few-shot-and-continual-learning-with-protection-from-forgetting-108095.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.38577>)

Author: David Rawlinson, Gideon Kowadlo

Published: 2026-10-30T04:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [few-shot](<https://devfeed.tech/topics/few-shot.md>), [few-shot-learning](<https://devfeed.tech/topics/few-shot-learning.md>), [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>)

Tags: [bias](<https://devfeed.tech/tags/bias.md>), [continual-learning](<https://devfeed.tech/tags/continual-learning.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [few-shot](<https://devfeed.tech/tags/few-shot.md>), [image-classification](<https://devfeed.tech/tags/image-classification.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>)

### AI overview

The proposed Complementary Learning System separates robust perception from short-term adaptation: a slow Long-Term Memory (LTM) produces generalized perceptions, while a Short-Term Memory (STM) memorizes recent inputs with less interference between memories. On an image classification task, this model resisted forgetting better than a baseline ResNet LTM in domain-incremental continual learning and generalized more effectively in few-shot learning than other continual learners. Its overall continual-learning accuracy was slightly lower than existing models, with a different mix of performance characteristics. The authors suggest LTM representations and their plasticity limit performance, and propose replaying STM memories to the LTM as future work.

### Source excerpt

arXiv:2610.38577v1 Announce Type: cross Abstract: There is a fundamental contradiction between the desire for models that can generalize, and models that can continuously adapt to recent input with changing statistics. These conditions, captured in few-shot and continual learning settings, can bias or disrupt learned representations. This harms generalization to a broader context, leaving the model unable to overcome input variance. This paper proposes a solution based on the Complementary Learning System architecture (CLS), in which a Short-Term Memory (STM) delegates generalization to a slow, statistical Long-Term Memory (LTM). Delegation is achieved by working in a space of robust, generalized perceptions produced by the LTM. With this separation, the STM can focus on memorizing recent input and avoiding interference between memories. We show that on an image classification task, the resulting CLS model is more resistant to forgetting in a domain-incremental continual learning setting than a baseline ResNet LTM, and generalizes more effectively in a few-shot setting than other continual learners. In the continual setting, overall accuracy is slightly worse than existing continual learning models, but with a different mix of performance characteristics. Results analysis suggests that the model is limited by the representations and representational plasticity of the LTM. In future work, we will seek to address this by replaying STM memories to the LTM.

## In-Context Multi-Operator Learning with DeepOSets

DevFeed: [In-Context Multi-Operator Learning with DeepOSets](<https://devfeed.tech/articles/in-context-multi-operator-learning-with-deeposets-103731.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2512.16074>)

Author: Shao-Ting Chiu, Aditya Nambiar, Ali Syed, Jonathan W. Siegel, Ulisses Braga-Neto

Published: 2026-10-28T04:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [AI for science](<https://devfeed.tech/topics/ai-for-science.md>), [Neural Network](<https://devfeed.tech/topics/neural-network.md>), [few-shot-learning](<https://devfeed.tech/topics/few-shot-learning.md>)

Tags: [architecture](<https://devfeed.tech/tags/architecture.md>), [attention](<https://devfeed.tech/tags/attention.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [budget](<https://devfeed.tech/tags/budget.md>), [comparison](<https://devfeed.tech/tags/comparison.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [error](<https://devfeed.tech/tags/error.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [flow](<https://devfeed.tech/tags/flow.md>), [inference](<https://devfeed.tech/tags/inference.md>), [latency](<https://devfeed.tech/tags/latency.md>), [learning](<https://devfeed.tech/tags/learning.md>), [operator](<https://devfeed.tech/tags/operator.md>), [operator-learning](<https://devfeed.tech/tags/operator-learning.md>), [operators](<https://devfeed.tech/tags/operators.md>), [pairs](<https://devfeed.tech/tags/pairs.md>), [parameter](<https://devfeed.tech/tags/parameter.md>), [partial-differential-equations](<https://devfeed.tech/tags/partial-differential-equations.md>), [scientific-computing](<https://devfeed.tech/tags/scientific-computing.md>), [time](<https://devfeed.tech/tags/time.md>), [universality](<https://devfeed.tech/tags/universality.md>)

### AI overview

DeepOSets is an attention-free, parameter-efficient architecture for learning multiple nonlinear operators, using a small set of example input-output pairs at inference to identify the desired operator. The authors give a mathematical formulation and prove universality for this task, then report experiments on Burgers' equation, 1D and 2D Darcy flow, and initial- and boundary-value differential equations. On 1D Darcy flow with 10k training steps, DeepOSets had lower error than transformer-based ICON at every tested context size, using 19% of its parameters. With 100k steps, ICON achieved 26-41% lower error, but used roughly 218 times more training time, 12 times higher inference latency, and 5.3 times more parameters.

### Source excerpt

arXiv:2512.16074v3 Announce Type: replace Abstract: An important application of neural networks in scientific computing is non-linear operator learning. In this framework, a neural network is trained to fit a non-linear map between infinite dimensional spaces, for example, the solution operator of ordinary and partial differential equations. Recently, inspired by the discovery of in-context learning for large language models, an even more ambitious paradigm has been explored, called multi-operator learning. In this approach, a neural network is trained to learn many different operators at the same time. At inference time, the network is passed a small number of example input and output pairs to disambiguate the desired operator. In this work, we provide a precise mathematical formulation of the multi-operator learning problem, propose a parameter-efficient, attention-free architecture, called DeepOSets, and prove its universality in the in-context multi-operator learning task. We perform experiments demonstrating the efficacy of DeepOSets on several benchmarks, including Burgers' equation, 1D and 2D Darcy flow, and a variety of initial-value and boundary-value differential equations. In a comparison with the transformer-based ICON model on the 1D Darcy problem, under a small training budget (10k steps), DeepOSets attained lower error at every context size with 19% of the parameters, while with an extended budget (100k steps), ICON attains 26-41% lower error, but at the cost of roughly 218$\times$ more training time, 12$\times$ higher inference latency, and 5.3$\times$ more parameters.

## Collaborative In-Context Learning for Indoor Radio Map Estimation

DevFeed: [Collaborative In-Context Learning for Indoor Radio Map Estimation](<https://devfeed.tech/articles/collaborative-in-context-learning-for-indoor-radio-map-estimation-99495.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.31702>)

Author: Yuanzhe Peng, Jie Xu

Published: 2026-10-26T04:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Neural Network](<https://devfeed.tech/topics/neural-network.md>), [few-shot-learning](<https://devfeed.tech/topics/few-shot-learning.md>)

Tags: [aggregation](<https://devfeed.tech/tags/aggregation.md>), [autoregressive](<https://devfeed.tech/tags/autoregressive.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [clients](<https://devfeed.tech/tags/clients.md>), [coverage](<https://devfeed.tech/tags/coverage.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [deployment](<https://devfeed.tech/tags/deployment.md>), [distributed](<https://devfeed.tech/tags/distributed.md>), [experimental](<https://devfeed.tech/tags/experimental.md>), [layouts](<https://devfeed.tech/tags/layouts.md>), [learning](<https://devfeed.tech/tags/learning.md>), [localization](<https://devfeed.tech/tags/localization.md>), [map](<https://devfeed.tech/tags/map.md>), [model-training](<https://devfeed.tech/tags/model-training.md>), [modeling](<https://devfeed.tech/tags/modeling.md>), [privacy](<https://devfeed.tech/tags/privacy.md>), [private](<https://devfeed.tech/tags/private.md>), [radio](<https://devfeed.tech/tags/radio.md>), [supervised-learning](<https://devfeed.tech/tags/supervised-learning.md>), [theoretical](<https://devfeed.tech/tags/theoretical.md>), [updates](<https://devfeed.tech/tags/updates.md>)

### AI overview

Poly-Ripple frames indoor radio map estimation as collaborative in-context learning, aiming to reduce reliance on labeled training data and centralized building information. Multiple clients use large autoregressive vision models without updating them; each provides private demonstrations to predict a radio map for the same scene query. Ripple-ordered autoregressive modeling represents propagation outward from the transmitter, while importance-aware aggregation combines clients' heterogeneous contextual knowledge under a context-length constraint. The authors report that experiments on a public indoor benchmark show consistent gains over representative in-context learning baselines, in line with their theoretical analysis.

### Source excerpt

arXiv:2610.31702v1 Announce Type: new Abstract: Indoor radio map estimation (RME) is essential for wireless applications such as coverage planning, localization, and network deployment. However, most existing methods follow a supervised learning paradigm that requires costly labeled data and task-specific model training. Moreover, indoor environments involve privacy-sensitive information, such as building layouts and structures, that often cannot be centralized. To address these challenges, we make the first attempt to formulate indoor RME as a collaborative in-context learning (ICL) problem. We propose Poly-Ripple, a framework in which multiple clients perform ICL using large autoregressive vision models (LAVMs) without model updates. Each client is prompted by its private demonstrations and predicts the radio map for the same scene query. To capture the causal structure of wireless propagation outward from the transmitter (Tx), we adopt ripple-ordered autoregressive modeling. We further introduce importance-aware aggregation to exploit heterogeneous contextual knowledge distributed across clients under the context length constraint. We provide theoretical analysis and conduct extensive experiments on a public indoor RME benchmark. The experimental results show that Poly-Ripple consistently outperforms representative ICL baselines, while aligning with our theoretical analysis.

## RheOPT: Few-Shot Non-Newtonian Flow Prediction via Newtonian Pretraining of RheOFormer

DevFeed: [RheOPT: Few-Shot Non-Newtonian Flow Prediction via Newtonian Pretraining of RheOFormer](<https://devfeed.tech/articles/rheopt-few-shot-non-newtonian-flow-prediction-via-newtonian-pretraining-of-rheoformer-97068.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.31465>)

Author: Maedeh Saberi, Amir Barati Farimani, Safa Jamali

Published: 2026-10-23T04:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [few-shot](<https://devfeed.tech/topics/few-shot.md>), [few-shot-learning](<https://devfeed.tech/topics/few-shot-learning.md>), [Computational Fluid Dynamics (CFD)](<https://devfeed.tech/topics/cfd.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [Neural Network](<https://devfeed.tech/topics/neural-network.md>), [Simulation](<https://devfeed.tech/topics/simulation.md>), [Inference](<https://devfeed.tech/topics/inference.md>)

Tags: [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [few-shot](<https://devfeed.tech/tags/few-shot.md>), [flow](<https://devfeed.tech/tags/flow.md>), [neural](<https://devfeed.tech/tags/neural.md>), [operator-learning](<https://devfeed.tech/tags/operator-learning.md>), [pretraining](<https://devfeed.tech/tags/pretraining.md>), [transformer](<https://devfeed.tech/tags/transformer.md>)

### AI overview

RheOPT pretrains a transformer neural operator on inexpensive Newtonian simulations, then fine-tunes it on a few non-Newtonian samples, reusing geometry-conditioned transport structure rather than Newtonian constitutive behavior. In a 4:1 contraction across Giesekus, Oldroyd-B, and Bingham fluids, four target samples matched or exceeded a from-scratch model trained on four times as many; the gain was reliable with reversible instance normalization (RevIN) but dataset-dependent with Global normalization, which the authors link to differences in velocity and stress dynamic ranges. For Oldroyd-B flow past a triangular obstacle, phase-insensitive diagnostics showed pretraining recovered wake shedding frequency and fluctuation amplitude suppressed by few-shot training from scratch. The evidence is limited to these two geometry-matched settings.

### Source excerpt

arXiv:2610.31465v1 Announce Type: new Abstract: Despite advances in numerical methods, non-Newtonian flow simulations remain computationally expensive because velocity and stress must be advanced together through rate- and history-dependent constitutive dynamics. Neural-operator surrogates reduce inference cost after training, but their training data require the same expensive simulations they aim to replace, making data-efficient learning essential when only a few non-Newtonian simulations can be afforded. Here, we introduce the Rheological Operator Pretrained Transformer (RheOPT), which pretrains a transformer-based neural operator on inexpensive Newtonian simulations and fine-tunes it on a small number of non-Newtonian samples. Rather than transferring the Newtonian constitutive response itself, RheOPT reuses geometry-conditioned transport structure that remains useful after the stress model changes. We evaluate RheOPT in two geometry-matched settings: a $4{:}1$ contraction channel for Giesekus, Oldroyd-B, and Bingham fluids, and the oscillatory wake behind a triangular obstacle for Oldroyd-B flow. In the contraction, Newtonian pretraining substantially improves few-shot prediction, and a model fine-tuned on four target samples can match or exceed a from-scratch model trained on four times as many. This benefit is reliable under reversible instance normalization (RevIN) but dataset dependent under Global normalization, a difference we trace to the dynamic range of each fluid's velocity and stress fields. In the wake, where relative $L_2$ error alone cannot verify that vortex shedding is recovered, phase-insensitive diagnostics show that pretraining recovers the shedding frequency and wake fluctuation amplitude that are suppressed by the from-scratch model in the few-shot regime. Overall, RheOPT establishes Newtonian simulation as a physically grounded pretraining source for data-efficient non-Newtonian operator learning.

## Bridging the Data Gap: Source-Informed Task-Space Transport for Few-Shot Diffusion

DevFeed: [Bridging the Data Gap: Source-Informed Task-Space Transport for Few-Shot Diffusion](<https://devfeed.tech/articles/bridging-the-data-gap-source-informed-task-space-transport-for-few-shot-diffusion-97050.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.31323>)

Author: Jinjian Jiang, Jing Peng, Rehan Deen, Yu Cao, Ziquan Liu, Muhammad Salman Haleem, David Mguni

Published: 2026-10-23T04:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [few-shot](<https://devfeed.tech/topics/few-shot.md>), [few-shot-learning](<https://devfeed.tech/topics/few-shot-learning.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [Representation learning](<https://devfeed.tech/topics/representation-learning.md>), [diffusion-transformers](<https://devfeed.tech/topics/diffusion-transformers.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [acquisition](<https://devfeed.tech/tags/acquisition.md>), [align](<https://devfeed.tech/tags/align.md>), [applying](<https://devfeed.tech/tags/applying.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [bridging](<https://devfeed.tech/tags/bridging.md>), [can](<https://devfeed.tech/tags/can.md>), [capacity](<https://devfeed.tech/tags/capacity.md>), [collect](<https://devfeed.tech/tags/collect.md>), [comparison](<https://devfeed.tech/tags/comparison.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [data](<https://devfeed.tech/tags/data.md>), [deployments](<https://devfeed.tech/tags/deployments.md>), [diffusion](<https://devfeed.tech/tags/diffusion.md>), [distribution](<https://devfeed.tech/tags/distribution.md>), [domain](<https://devfeed.tech/tags/domain.md>), [few-shot](<https://devfeed.tech/tags/few-shot.md>), [gap](<https://devfeed.tech/tags/gap.md>), [source](<https://devfeed.tech/tags/source.md>), [space](<https://devfeed.tech/tags/space.md>), [transport](<https://devfeed.tech/tags/transport.md>)

### AI overview

Meta-Diff adapts a diffusion generator to a sparse target by encoding source and target distributions into a compact task space, learning a reusable source-to-target relation, and generating from the transported source coordinate without target-specific retraining. In controlled Gaussian-mixture experiments, success depends on preserving task identity, generalizing the relation, and having useful headroom for transport. On held-out CIFAR-100 validation tasks, correct-source transport had 13.9% lower mean SW than transport from the wrong source. In a final-test one-target comparison using validation-selected settings, it was 14.7% lower than target-only modeling and 32.7% lower than full fine-tuning; the main route uses no target image and assumes the applicable relation is known. Limits are evident: on matched CIFAR test episodes, clean-source generation followed by known corruption had lower SW than transport, while Office-Home showed that domain statistics could align without reliable object identity.

### Source excerpt

arXiv:2610.31323v1 Announce Type: new Abstract: Many deployments require a model to move rapidly from a data-rich source to a related target observed only sparsely. This is especially consequential where target data are difficult to collect, as in healthcare settings with persistent population or acquisition imbalances. Retraining or fine-tuning a high-capacity generator for every target is costly and can be statistically unreliable when target data are scarce. We introduce Meta-Diff, which instead learns a recurring source--target relation once in a compact task space. A shared encoder maps each distribution to a low-dimensional coordinate; a learned relation transports a new source coordinate to its predicted target, and a shared diffusion backbone generates from that prediction without target-specific retraining. Controlled Gaussian-mixture experiments identify when this works: task identity must survive the representation, the relation must generalise across tasks, and the source--target shift must leave useful headroom for transport. On held-out CIFAR-100 validation tasks, correct source $+$ transport achieves 13.9% lower mean SW than transport from the wrong source. With validation-selected adaptation settings in the final-test one-target comparison, its mean SW is 14.7% lower than target-only modelling and 32.7% lower than full fine-tuning; the main route itself uses no target image and assumes the applicable relation is known. On matched CIFAR test episodes, generating from the clean-source coordinate and then applying the known corruption yields lower SW than task-space transport. Office-Home exposes a complementary boundary: transport can align target-domain statistics without reliable object identity. Together, these results recast extreme-few-shot adaptation as reusable transport in task space, while making the conditions for its success and failure directly measurable.

## Weight Space Learning for Certifiable Few-shot Transfer Learning

DevFeed: [Weight Space Learning for Certifiable Few-shot Transfer Learning](<https://devfeed.tech/articles/weight-space-learning-for-certifiable-few-shot-transfer-learning-94067.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2502.06970>)

Author: Fady Rezk, Royson Lee, Henry Gouk, Timothy Hospedales, Minyoung Kim

Published: 2026-10-21T04:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [few-shot](<https://devfeed.tech/topics/few-shot.md>), [few-shot-learning](<https://devfeed.tech/topics/few-shot-learning.md>), [peft](<https://devfeed.tech/topics/peft.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [AI Foundation Models](<https://devfeed.tech/topics/ai-foundation-models.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>)

Tags: [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [diffusion](<https://devfeed.tech/tags/diffusion.md>), [few-shot](<https://devfeed.tech/tags/few-shot.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [foundation-models](<https://devfeed.tech/tags/foundation-models.md>), [guarantees](<https://devfeed.tech/tags/guarantees.md>), [risk](<https://devfeed.tech/tags/risk.md>), [spaces](<https://devfeed.tech/tags/spaces.md>), [stat-ml](<https://devfeed.tech/tags/stat-ml.md>), [transfer-learning](<https://devfeed.tech/tags/transfer-learning.md>)

### AI overview

The paper proposes few-shot transfer learning that aims to provide non-vacuous generalization certificates. It trains a diffusion model on upstream tasks to learn a distribution over parameter-efficient fine-tuning (PEFT) parameters, then samples candidate PEFTs and selects the one with the highest likelihood on downstream data. Restricting the downstream hypothesis space to a finite set of samples is intended to enable tighter risk certificates than continuous neural-network weight spaces. The authors report non-trivial guarantees in the low-shot regime, where the compared learning approaches produce vacuous bounds.

### Source excerpt

arXiv:2502.06970v3 Announce Type: replace Abstract: In contemporary deep learning, a prevalent and effective workflow for solving low-data problems is adapting powerful pre-trained foundation models (FMs) to new tasks via parameter-efficient fine-tuning (PEFT). However, while empirically effective, the resulting solutions lack generalisation guarantees to certify their accuracy - which may be required for ethical or legal reasons prior to deployment in high-importance applications. In this paper we develop a novel transfer learning approach that is designed to facilitate non-vacuous learning theoretic generalisation guarantees for downstream tasks, even in the low-shot regime. Specifically, we first use upstream tasks to train a distribution over PEFT parameters. We then learn the downstream task by a sample-and-evaluate procedure -- sampling plausible PEFTs from the trained diffusion model and selecting the one with the highest likelihood on the downstream data. Crucially, this confines our model hypothesis to a finite set of PEFT samples. In contrast to the typical continuous hypothesis spaces of neural network weights, this facilitates tighter risk certificates. We instantiate our bound and show non-trivial generalization guarantees compared to existing learning approaches which lead to vacuous bounds in the low-shot regime.

## Meta-iLaD: Identifiable Latent Dynamics via Meta-Learning of Dynamics Environments

DevFeed: [Meta-iLaD: Identifiable Latent Dynamics via Meta-Learning of Dynamics Environments](<https://devfeed.tech/articles/meta-ilad-identifiable-latent-dynamics-via-meta-learning-of-dynamics-environments-94251.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.27180>)

Author: Yubo Ye, Sweekar Piya, Xiajun Jiang, Linwei Wang

Published: 2026-10-21T04:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [few-shot](<https://devfeed.tech/topics/few-shot.md>), [few-shot-learning](<https://devfeed.tech/topics/few-shot-learning.md>), [generalization in machine learning](<https://devfeed.tech/topics/generalization-in-machine-learning.md>), [Time Series](<https://devfeed.tech/topics/time-series.md>), [Predictive AI](<https://devfeed.tech/topics/predictive-ai.md>)

Tags: [arxiv](<https://devfeed.tech/tags/arxiv.md>), [code](<https://devfeed.tech/tags/code.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [distribution](<https://devfeed.tech/tags/distribution.md>), [environments](<https://devfeed.tech/tags/environments.md>), [few-shot](<https://devfeed.tech/tags/few-shot.md>), [forecasting](<https://devfeed.tech/tags/forecasting.md>), [learning](<https://devfeed.tech/tags/learning.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [meta](<https://devfeed.tech/tags/meta.md>), [synthetic](<https://devfeed.tech/tags/synthetic.md>), [time-series](<https://devfeed.tech/tags/time-series.md>)

### AI overview

Meta-iLaD targets latent dynamics models that rely on predefined environment labels and leave the joint identifiability of the latent state, environment, and dynamics function unclear. It meta-learns across dynamics environments and uses a feedforward conditional prior to infer an environment variable from few-shot examples. The authors state that their identifiability results cover the latent state, environment variable, and dynamics function without restricting the environment variable's dimension or how it modulates the dynamics. On synthetic and real data, they report evidence that few-shot conditioning supports generalization to out-of-distribution environments, and that identifying the environment and dynamics function matters for accurate forecasting beyond reconstructing observed trajectories.

### Source excerpt

arXiv:2610.27180v1 Announce Type: new Abstract: Learning latent dynamics is central to assessing the current states and forecasting the future trajectories of high-dimensional time series. For locally-stationary latent dynamics $\mathcal{F}(\mathbf{z}_{< t};\mathbf{c})$ without restricting the dimension of $\mathbf{c}$ or how it modulates $\mathcal{F}$. On synthetic and real data, we provide strong empirical evidence that 1) conditioning on few-shot examples enables generalization to out-of-distribution environments, and 2) identifiability for $\mathbf{c}$ and $\mathcal{F}$ is critical for accurate forecasting beyond reconstructing observed trajectories. Code is available at https://github.com/GoForIt-YYB/Meta-iLaD.

[Next page](<https://devfeed.tech/topics/few-shot-learning.md?cursor=WyIyMDI2LTEwLTIxVDA0OjAwOjAwKzAwOjAwIiwgIjA0ZmI0NzQzLTM0YTAtNDdmMC1iZjA1LWE3MTlhZDAzODUzMCJd>)