# backpropagation Published articles for backpropagation. This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## End-to-End On-Device Quantization-Aware Training for LLMs at Inference Cost DevFeed: [End-to-End On-Device Quantization-Aware Training for LLMs at Inference Cost]() Original publisher: [Read original article]() Author: Qitao Tan, Xiaoying Song, Ningxi Cheng, Guoming Li, Jun Liu, Lingzi Hong, Caiwen Ding, Jundong Li, Xiaoming Zhai, Shaoyi Huang, Jin Lu, Ninghao Liu, Wei Niu, Geng Yuan Published: 2026-11-09T05:00:00Z Content type: article Language: en Sources: [arXiv: Machine Learning]() Topics: [quantization](), [LLM optimization](), [LLM Techniques](), [Fine-tuning](), [Large Language Model]() Tags: [activation](), [backpropagation](), [bits](), [cs-ai](), [cs-lg](), [deployment](), [device](), [edge](), [edge-devices](), [fine-tuning](), [gpu](), [gradient](), [inference](), [large-language-models-llms](), [lightweight](), [llms](), [memory](), [on-device](), [oneplus](), [optimization](), [quantization](), [training]() ### AI overview ZeroQAT applies quantization-aware training to LLM weights and activations using forward-only, zeroth-order gradient estimates. This avoids backpropagation and is reported to reduce memory and computation while preserving end-to-end optimization. A lightweight fine-tuning variant freezes and pre-quantizes most parameters to reduce memory further. The abstract reports that ZeroQAT outperforms representative post-training quantization and QAT baselines while using less memory; it cites fine-tuning a 13B model at 2-4 bits on one 8GB GPU and a 6.7B model on a OnePlus 12 smartphone. These are abstract-level reported results; the supplied excerpt gives no evaluation details. ### Source excerpt arXiv:2509.00031v3 Announce Type: replace Abstract: Quantization reduces the deployment cost of large language models (LLMs). However, post-training quantization (PTQ) cannot fine-tune model parameters and often degrades accuracy in low-bit settings, while quantization-aware training (QAT) requires backpropagation and incurs high memory overhead. We propose ZeroQAT, a zeroth-order QAT framework supporting both weight and activation quantization. By using forward-only gradient estimation, ZeroQAT removes backpropagation and substantially reduces memory and computation while retaining end-to-end optimization benefits. We further introduce a lightweight variant of ZeroQAT for quantized fine-tuning, which freezes and pre-quantizes most parameters to further cut memory usage. Experiments show that ZeroQAT consistently outperforms representative PTQ and QAT baselines while requiring significantly less memory. For example, ZeroQAT enables fine-tuning of a 13B model at extremely low bit-widths (e.g., 2-4 bits) on a single 8GB GPU, and even allows fine-tuning a 6.7B model on a OnePlus 12 smartphone, demonstrating its practicality for end-to-end QAT on resource-limited edge devices. ## Stability and Generalization in Looped Transformers DevFeed: [Stability and Generalization in Looped Transformers]() Original publisher: [Read original article]() Author: Asher Labovich Published: 2026-11-06T05:00:00Z Content type: article Language: en Sources: [arXiv: Artificial Intelligence](), [arXiv: Machine Learning]() Topics: [Transformers](), [Model Development](), [scaling laws]() Tags: [backpropagation](), [chess](), [cs-ai](), [cs-lg](), [fixed](), [framework](), [input](), [recall](), [tasks](), [transformers]() ### AI overview The paper argues that useful test-time extrapolation in looped transformers depends on fixed-point stability along three axes: local attractivity, input-dependence, and geometry. It proves that networks without recall have countable fixed points and cannot achieve strong input-dependence across spectral regimes. Recall with outer normalization, by contrast, supports a regime where fixed points are locally attractive and smooth in the input, with stable backpropagation. Experiments on chess, sudoku, and prefix-sums report that downstream performance tracks these predictions across tasks and architectural configurations. The authors also introduce internal recall, which permits an identity residual path, and use its contrasting fixed-point behavior to test the framework. ### Source excerpt arXiv:2604.15259v3 Announce Type: replace-cross Abstract: Looped transformers promise test-time compute scaling by spending more iterations on harder problems, but it remains unclear which architectural choices let them extrapolate to harder problems at test time rather than memorize training-specific solutions. We introduce a fixed-point based framework for analyzing looped architectures along three axes of stability -- local attractivity, input-dependence, and geometry -- and use it to characterize when fixed-point iteration yields meaningful predictions. Theoretically, we prove that looped networks without recall have countable fixed points and cannot achieve strong input-dependence at any spectral regime, while recall with outer normalization reliably produces a regime in which fixed points are simultaneously locally-attractive, locally smooth in the input, and supported by stable backpropagation. Empirically, we train small looped transformers on chess, sudoku, and prefix-sums and find that downstream performance tracks the framework's predictions across tasks and architectural configurations. We additionally introduce internal recall, a novel recall placement variant allowing an identity residual path, and exploit its contrasting fixed-point behavior to test our framework's predictions in a controlled setting. ## Grokking or Glitching? How Low-Precision Drives Slingshot Loss Spikes DevFeed: [Grokking or Glitching? How Low-Precision Drives Slingshot Loss Spikes]() Original publisher: [Read original article]() Author: Hanqing Liu, Jianjun Cao, Yuanze Li, Zijian Zhou Published: 2026-11-06T05:00:00Z Content type: article Language: en Sources: [arXiv: Machine Learning]() Topics: [floating-point](), [Deep neural networks](), [Artificial Intelligence]() Tags: [absorption](), [backpropagation](), [constraint](), [cs-cl](), [cs-lg](), [deep-neural-networks](), [feature](), [feedback](), [floating-point](), [growth](), [inflation](), [math-oc](), [optimization](), [parameter](), [precision](), [stat-ml](), [training]() ### AI overview The paper attributes Slingshot loss spikes to finite-precision arithmetic. In high-confidence training, the correct-class logit can exceed an absorption-error threshold, causing its gradient to round to zero while incorrect-class gradients remain nonzero. This breaks the gradients' zero-sum constraint and creates classifier-update drift. The authors call the resulting positive feedback with features Numerical Feature Inflation (NFI): classifier and feature means grow exponentially, norms rise, gradients later reappear, and a loss spike can follow. NFI is not equivalent to a visible spike; partial absorption may drive rapid parameter-norm growth without an observable loss spike. ### Source excerpt arXiv:2605.06152v5 Announce Type: replace Abstract: Deep neural networks exhibit periodic loss spikes during unregularized long-term training, a phenomenon known as the "Slingshot Mechanism." Existing work usually attributes this to intrinsic optimization dynamics, but its triggering mechanism remains unclear. This paper proves that this phenomenon is a result of floating-point arithmetic precision limits. As training enters a high-confidence stage, the difference between the correct-class logit and the other logits may exceed the absorption-error threshold. Then during backpropagation, the gradient of the correct class is rounded exactly to zero, while the gradients of the incorrect classes remain nonzero. This breaks the zero-sum constraint of gradients across classes and introduces a systematic drift in the parameter update of the classifier layer. We prove that this drift forms a positive feedback loop with the feature, causing the global classifier mean and the global feature mean to grow exponentially. We call this mechanism Numerical Feature Inflation (NFI). This mechanism explains the rapid norm growth before a Slingshot spike, the subsequent reappearance of gradients, and the resulting loss spike. We further show that NFI is not equivalent to an observed loss spike: in more practical tasks, partial absorption may not produce visible spikes, but it can still break the zero-sum constraint and drive rapid growth of parameter norms. Our results reinterpret Slingshot as a numerical dynamic of finite-precision training, and provide a testable explanation for abnormal parameter growth and logit divergence in late-stage training. ## Training Specialist LLMs on Your Phone: Forward-Only Optimization via Block-Spectral Coordinates DevFeed: [Training Specialist LLMs on Your Phone: Forward-Only Optimization via Block-Spectral Coordinates]() Original publisher: [Read original article]() Author: Liyan Tan, Yequan Zhao, Ziyue Liu, Yupeng Su, Ruijie Zhang, Zheng Zhang Published: 2026-11-04T05:00:00Z Content type: article Language: en Sources: [arXiv: Machine Learning]() Topics: [rlvr](), [Large Language Model](), [Machine learning](), [Fine-tuning](), [Reinforcement learning](), [quantization](), [LLM optimization](), [Hyperparameter optimization]() Tags: [agent](), [algorithm](), [backpropagation](), [benchmarks](), [blocks](), [column](), [cs-lg](), [cs-sy](), [device](), [eess-sy](), [es](), [int4](), [iphone](), [iphone-16-pro](), [language-models](), [large-language-models](), [large-language-models-llms](), [learning](), [lightweight](), [llms](), [optimization](), [phone](), [training]() ### AI overview MobileSpec trains lightweight, personalized on-device LLM specialists using forward passes and automatically checked task outcomes, avoiding backpropagation and labeled examples. Its FuRA-S method updates only block-spectral coordinates--the singular values of weight column blocks--whose independent rescaling rotates singular vectors. The update can also be expressed as a layer-input transformation, allowing adaptation of INT4 models without changing their weights. The abstract reports 1-5 MB specialists updating 0.015-0.05% of parameters, phone-use agent success rising from 6-23% to 89-100%, and overnight personalization of an 8B model on an iPhone 16 Pro. On RLVR benchmarks, FuRA-S is reported as competitive with full-parameter evolution strategies and better than ES over LoRA adapters and quantized ES on INT4 models. ### Source excerpt arXiv:2611.04797v1 Announce Type: new Abstract: Large language models (LLMs) on personal devices keep user data local, but limited memory restricts their general capability. Since most user-specific tasks do not require the breadth of frontier models, a practical solution is to adapt a shared on-device model into lightweight specialists. However, backpropagation is prohibitive in memory, and users rarely provide labeled data. We present MobileSpec, a system that trains specialists with forward passes alone and learns from automatically checked task outcomes, as in reinforcement learning with verifiable rewards (RLVR). Because forward-only updates are built from random perturbations, training is efficient only when these perturbations resemble the weight changes of RLVR, which largely preserve the singular values of pretrained weights and rotate their singular vectors. The core algorithm of MobileSpec, FuRA-S, produces such perturbations while training only the block-spectral coordinates of pretrained weights, namely the singular values of their column blocks. As we show theoretically and empirically, rescaling these blocks independently shifts their relative contributions to each weight matrix and thereby rotates its singular vectors. Since this update can be rewritten as a transformation of the layer input, FuRA-S also adapts INT4 models without modifying their weights. With specialists of only 1-5 MB that update 0.015-0.05% of the parameters, MobileSpec raises the success rate of a phone-use agent from 6-23% up to 89-100% and trains a personalized 8B model on an iPhone 16 Pro overnight. On RLVR benchmarks, FuRA-S is competitive with full-parameter evolution strategies (ES) and outperforms ES over LoRA adapters and quantized ES on INT4 models. ## CAPSkip: Cancellation-Aware Prefix Skipping for Selective Backpropagation DevFeed: [CAPSkip: Cancellation-Aware Prefix Skipping for Selective Backpropagation]() Original publisher: [Read original article]() Author: Heejin Choi Published: 2026-11-04T05:00:00Z Content type: article Language: en Sources: [arXiv: Machine Learning]() Topics: [LLM optimization](), [AI Infrastructure]() Tags: [backpropagation](), [cancellation](), [cs-lg](), [gradient](), [machine-learning]() ### AI overview CAPSkip reduces selective backpropagation by skipping a prefix of already-correct examples, ordered by last-layer gradient norm, while keeping the prefix's summed gradient within a relative budget of the batch's last-layer gradient. It scans every prefix because cumulative distortion is not monotone, using a factorised Gram matrix to make the scan cheap and keep it on-device. In exact arithmetic, the method bounds the change to the raw final-layer gradient using contributions available from the forward pass; this guarantee does not cover deeper layers, optimizer updates or the training trajectory. With activation reuse and selected-row backward, DeiT-S fine-tuning on CIFAR-100 took 0.576 times the time of 103-epoch dense training. Its result was 0.25 percentage points above an approximately time-matched 59-epoch dense run, with a reported interval of +0.08 to +0.42 pp across three runs. ### Source excerpt arXiv:2611.04611v1 Announce Type: new Abstract: Per-step selective back-propagation forwards every example of a mini-batch but back-propagates only some of them. Most rules rank examples by loss or gradient magnitude, but such scalar scores ignore cancellation between per-example gradients: the gradient removed by a set can be much smaller than its individual magnitudes suggest. We introduce CAPSkip (Cancellation-Aware Prefix Skipping). Among the examples the model already classifies correctly, CAPSkip skips the longest prefix, in increasing order of last-layer gradient norm, whose vector sum stays within a relative budget $b$ of the batch's last-layer gradient. In exact arithmetic this bounds the change of the raw final-layer gradient, using rank-one contributions already available from the forward pass; the bound does not extend to deeper layers, optimizer updates, or the training trajectory. Because the cumulative distortion is not monotone in the prefix length, CAPSkip scans all prefixes; a factorised Gram matrix makes the scan cheap and keeps it on the device. With activation reuse and selected-row backward, CAPSkip fine-tunes DeiT-S on CIFAR-100 for 103 epochs in 0.576x the time of 103-epoch dense training, and is +0.25 pp [+0.08, +0.42] above an approximately time-matched 59-epoch dense run (n=3). ## Which Chains to Calibrate for Physical Backpropagation: Per-User Factors Versus Base-Station Chains DevFeed: [Which Chains to Calibrate for Physical Backpropagation: Per-User Factors Versus Base-Station Chains]() Original publisher: [Read original article]() Author: Xianzhong Li, Yuzhi Yang, Tianhao Guo Published: 2026-11-03T05:00:00Z Content type: article Language: en Sources: [arXiv: Machine Learning]() Topics: [communications](), [fourier analysis]() Tags: [amplitude](), [backpropagation](), [channel](), [cs-it](), [cs-lg](), [echo](), [eess-sp](), [estimator](), [factor](), [gradient](), [independent](), [iteration](), [math-it](), [noise](), [oracle](), [ordering](), [rotation](), [selection](), [service](), [serving](), [signal](), [size](), [snr]() ### AI overview In physical backpropagation, users send loss derivatives over the TDD uplink so the base station can obtain the downlink gradient without reconstructing the channel. The abstract finds that per-user calibration can matter more than base-station calibration: user amplitude errors can distort the returned field and drive wrong-user selection, while user phase errors mainly slow iteration. Base-station chain errors scale the field per antenna and can impose an SNR penalty that array size does not remove. In one 80-user clustered-channel setting with errors on both sides and mismatched pilots, oracle per-user calibration left 6.2% loss at 256 antennas, versus 12.3% for oracle base-station calibration; the ordering held at 30 dB SNR but reversed for 1 dB, 10-degree user errors. A capped one-echo estimator approached oracle per-user calibration within 2.4 percentage points, yet performed worse than no calibration at 10 dB SNR. ### Source excerpt arXiv:2611.02390v1 Announce Type: cross Abstract: In physical backpropagation, users return loss derivatives over the time-division duplex uplink, giving the base station the downlink gradient without channel reconstruction. Reciprocity calibration conventionally prioritizes base-station chains, as column-normalized zero-forcing reduces a per-user factor to a stream rotation. This priority can reverse. With ideal base-station chains and no noise, per-user amplitude errors make the returned field a weighted sum-rate gradient. With zero-forcing full service at both optima, the loss is independent of array size. Otherwise the selection penalty, serving the wrong users, carries 86 to 94 percent of the amplitude-induced loss in five of six configurations. Per-user phase errors mainly slow the iteration. Base-station chain errors scale the returned field per antenna, and at 3 dB and 25 degrees they cost one user 1.3 dB of signal-to-noise ratio (SNR) that no array size removes. With 80 users, such errors on both sides, and mismatched pilots, oracle per-user and base-station calibration leave 6.2 and 12.3 percent loss at 256 antennas on clustered channels. This ordering holds at 30 dB SNR and reverses for 1 dB, 10 degree user errors. A capped one-echo estimator comes within 2.4 percentage points of oracle per-user calibration but underperforms no calibration at 10 dB SNR. ## Nearest Is Not Safest: Fluid-Antenna Port Selection for Over-the-Air Backpropagation in Urban Multipath DevFeed: [Nearest Is Not Safest: Fluid-Antenna Port Selection for Over-the-Air Backpropagation in Urban Multipath]() Original publisher: [Read original article]() Author: Xianzhong Li, Yuzhi Yang, Tianhao Guo Published: 2026-11-03T05:00:00Z Content type: article Language: en Sources: [arXiv: Machine Learning]() Topics: [communications]() Tags: [angular](), [antenna](), [backpropagation](), [certificate](), [channel](), [cities](), [cs-it](), [cs-lg](), [eess-sp](), [gaussian](), [gradient](), [history](), [knowledge](), [linear](), [math-it](), [mirror](), [model](), [port](), [selection](), [survey](), [virtual]() ### AI overview In ray-traced Chicago streets, choosing the fluid-antenna port nearest the forward position reversed 20% of gradients at 0.7-wavelength spacing and 49% at 0.9 wavelengths; among the four nearest ports, misalignment was only 2% and 9%. A knowledge map using surveyed paths, virtual sources, and causal channel history cut reversals at 0.9 wavelengths to 27% at unsurveyed positions in the same city. Its advantage over the nearest-port rule persisted across tested errors, while its advantage over a geometry rule held at tested single-error levels of 5° phase drift per step and 2 mm position error, but not at the next tested levels. Without surveys, across five cities and ten base stations, a base-station direction-and-elevation rule had a 20.5% reversal rate versus 37% for nearest-port selection. Tested multilayer perceptrons did not beat the physical rules by the preset margin in primary settings. The authors also derive a tight certificate for mean-correlation sign from planar angular-spectrum second moments; in a static Gaussian model, line observations alone cannot determine the sign of the cross moment. ### Source excerpt arXiv:2611.02384v1 Announce Type: cross Abstract: Over-the-air backpropagation returns a loss gradient through the reciprocal channel, which requires the return port to see the channel of the forward port. If the terminal moves in between, a fluid antenna system can return through the port nearest to the forward position. In ray-traced Chicago streets, with a pedestrian displacement of about half a wavelength and a street-level base station, this rule reverses 20 percent of the gradients at a port spacing of 0.7 wavelengths and 49 percent at 0.9 wavelengths, while one of the four nearest ports stays aligned in all but 2 and 9 percent. We derive a certificate for the sign of the mean correlation from the second moment of a planar angular spectrum. It reduces to a linear program with three variables and infinitely many constraints, is tight, and holds over a region of second moments. In a static Gaussian model, observations collected along a line cannot tell an angular spectrum from its mirror image, so the sign of the cross moment must come from other information. At unsurveyed positions of the same city, a knowledge map that transports surveyed paths through virtual sources and fits them to the causal channel history lowers the reversal rate at 0.9 wavelengths from 49 to 27 percent. Its gain over the nearest port survived every tested error, while its gain over a geometry rule stayed significant at the tested single-error levels of 5 degrees of phase drift per step and 2 mm of position error but not at the next tested levels. Without any survey, across five cities and ten base stations, the nearest port reverses 37 percent of the returns at 0.9 wavelengths and a rule based on the direction and elevation of the base station 20.5 percent. The multilayer perceptrons we tested did not beat these physical rules by the margin fixed in advance in the primary settings. ## Backpropagation-free Deep Learning for Time-Series Classification DevFeed: [Backpropagation-free Deep Learning for Time-Series Classification]() Original publisher: [Read original article]() Author: Miha Kap\\v{s}, Bla\\v{z} Bertalani\\v{c}, Carolina Fortuna Published: 2026-11-02T05:00:00Z Content type: article Language: en Sources: [arXiv: Artificial Intelligence](), [arXiv: Machine Learning]() Topics: [Deep learning](), [Machine learning](), [Time Series](), [Deep neural networks](), [Hyperparameter optimization]() Tags: [alignment](), [analysis](), [architecture](), [backpropagation](), [benchmarks](), [classification](), [cs-ai](), [cs-lg](), [datasets](), [deep](), [deep-learning](), [dfa](), [difference-in-differences](), [diffusion](), [feature](), [features](), [feedback](), [floating-point](), [framework](), [gap](), [learning](), [time-series]() ### AI overview Across 112 UCR time-series datasets, adapted Direct Feedback Alignment (DFA), Direct Random Target Projection (DRTP), and NoProp achieved mean F1 scores of 0.76-0.79, below ROCKET (0.83) and InceptionTime (0.84). Intra-class temporal variability was the dataset property most consistently associated with the performance gap. Global average pooling hurt DFA and DRTP more than a backpropagation control, while NoProp tolerated it similarly to backpropagation. Relative to backpropagation on the same architectures, DRTP cut training FLOPs by 33% but trained faster only for series longer than about 700 time steps; DFA reduced computation by none of the reported measures; NoProp cut peak memory by 38% while taking 48% longer to train. ### Source excerpt arXiv:2610.41805v1 Announce Type: new Abstract: Deep neural networks for time series classification (TSC) are trained almost exclusively by error backpropagation, which imposes update locking and the weight-transport problem. Backpropagation-free learning rules such as Direct Feedback Alignment (DFA), Direct Random Target Projection (DRTP), and the diffusion-inspired NoProp relax these constraints but have been evaluated almost exclusively on 2D image benchmarks. We adapt these three learners to univariate TSC with 1D convolutional architectures and present the first systematic study of them across 112 datasets from the UCR time series archive, with one hyperparameter-optimized configuration per method. The optimized learners reach mean F1 scores of 0.76 to 0.79, below ROCKET (0.83) and InceptionTime (0.84). We propose an analysis framework that relates the per-dataset gap to InceptionTime to dataset properties and tests convolutional feature learning by replacing the flatten readout with global average pooling, measured as a difference-in-differences (DD) against a backpropagation control on the identical architecture. Intra-class temporal variability is the dataset property most consistently associated with the gap for all three learners (partial Spearman $\rho$ from $0.28$ to $0.40$). Pooling harms DFA and DRTP more than backpropagation (DD $-0.025$ and $-0.055$), consistent with near-random convolutional features, whereas NoProp tolerates pooling as backpropagation does (DD $+0.013$). Relative to backpropagation on the same architectures, DRTP reduces training floating-point operations (FLOPs) by 33% and trains faster only on series longer than about 700 time steps, DFA yields no computational reduction, and NoProp reduces peak memory by 38% at the cost of 48% longer training. ## Binary Sparse Codes: Where are we on Energy Efficient Representations? DevFeed: [Binary Sparse Codes: Where are we on Energy Efficient Representations?]() Original publisher: [Read original article]() Author: Jose Miguel Ramos, Luis SaCouto, Andreas Wichert Published: 2026-11-02T05:00:00Z Content type: article Language: en Sources: [arXiv: Machine Learning]() Topics: [Representation learning](), [Artificial Intelligence](), [Algorithms]() Tags: [abstract](), [activation](), [also](), [apply](), [approach](), [architecture](), [arxiv](), [backpropagation](), [but](), [capabilities](), [challenge](), [code](), [coding](), [compare](), [cost](), [cs-lg](), [deep](), [deep-learning](), [efficiency](), [energy](), [evaluation]() ### AI overview The abstract proposes metrics that assess how well binary sparse codes preserve input relationships, then uses them to compare four representative methods under matched conditions. It reports a trade-off: stronger similarity preservation generally comes with lower sparsity, while codes closer to an optimal random sparse code tend to lose preserved information. Competitive Sparse Codes, a biologically plausible method using local Hebbian learning, is reported as the only architecture combining near-maximal sparsity with meaningful information preservation. The authors present simultaneous achievement of both properties as an open challenge; the evidence supplied is the abstract, not the full paper. ### Source excerpt arXiv:2610.41933v1 Announce Type: new Abstract: Biological neural systems achieve impressive computational capabilities under tight metabolic constraints, relying on sparse cortical representations that minimise total neural activity. This combination of efficiency and representational power has motivated growing interest in sparse coding as a computational strategy for artificial systems. Artificial models designed to produce binary sparse codes span three broad paradigms, theoretical methods grounded in efficient coding, deep learning methods trained via backpropagation, and biologically plausible methods that learn through local Hebbian rules, each yielding codes whose quality depends not only on achieving low activation density but also on preserving the informational structure of the input. The latter property, however, lacks an agreed-upon quantitative definition, leaving the field without a principled basis for comparing methods. We propose a set of metrics that formalise information preservation as similarity preservation, measuring how faithfully a binary sparse code reproduces the relational geometry of the input space, and apply this framework to compare four representative methods under matched conditions. The results reveal a consistent trade-off in which models that achieve high information preservation do so at the cost of sparsity, while models approaching the uniformity of an optimal random sparse code tend to sacrifice it. Competitive Sparse Codes, a biologically plausible approach, is the only architecture to occupy a region of near-maximal sparsity while retaining meaningful information preservation. These findings frame the simultaneous satisfaction of both factors as an open challenge for the field and provide a reusable evaluation framework for measuring progress toward it. ## How Much Is One Recurrence Worth? Iso-Depth Scaling Laws for Looped Language Models DevFeed: [How Much Is One Recurrence Worth? Iso-Depth Scaling Laws for Looped Language Models]() Original publisher: [Read original article]() Author: Kristian Schwethelm, Daniel Rueckert, Georgios Kaissis Published: 2026-10-30T04:00:00Z Content type: article Language: en Sources: [arXiv: Machine Learning]() Topics: [scaling laws](), [Large Language Model](), [LLM Techniques]() Tags: [backpropagation](), [capacity](), [case-studies](), [compute](), [cs-cl](), [cs-lg](), [language-models](), [pretraining](), [scaling](), [scaling-laws](), [validation]() ### AI overview An iso-depth pretraining sweep across recurrence counts 1, 2, 4, and 8 yields a recurrence-equivalence exponent of 0.46: shared recurrences provide some, but less than full, capacity equivalence to unique transformer blocks. At four recurrences, a 410M looped model matches a 580M non-looped model while costing as much to train as a 1B non-looped model. The exponent also distinguishes training changes: truncated backpropagation lowers it to 0.38 despite reducing validation loss, suggesting poorer training of the loop mechanism; hyperconnections raise it to 0.65, which the authors interpret as a capacity gain. The abstract presents this measure as a way to separate capacity improvements from training-side gains that raw validation loss cannot distinguish. ### Source excerpt arXiv:2604.21106v4 Announce Type: replace Abstract: We measure how much one recurrence is worth to a looped transformer in equivalent unique parameters. From an iso-depth pretraining sweep across recurrence counts $r \in \{1, 2, 4, 8\}$ spanning ${\sim}50\times$ in training compute, we fit a joint scaling law $L = E + A\,(N_\text{once} + r^{\varphi} N_\text{rec})^{-\alpha} + B\,D^{-\beta}$ and measure a recurrence-equivalence exponent $\varphi = 0.46$. Intuitively, $\varphi$ tells us whether looping a block $r$ times is equivalent in validation loss to $r$ unique blocks of a non-looped model (full equivalence, $\varphi{=}1$) or to a single block run repeatedly with no capacity gain ($\varphi{=}0$). Our $\varphi = 0.46$ sits in between, so replacing unique blocks with shared recurrences increases validation loss at matched training compute. For example, at $r{=}4$ a 410M looped model performs on par with a 580M non-looped model, but incurs the training cost of a 1B non-looped one. We demonstrate the utility of $\varphi$ as a diagnostic tool on two case studies: commonly used truncated backpropagation lowers $\varphi$ to $0.38$, indicating that the loop mechanism is poorly trained under truncation, even though validation loss decreases. Conversely, hyperconnections raise $\varphi$ to $0.65$, a genuine capacity gain. Our method separates true loop improvements from training-side gains, a distinction raw validation loss cannot make. Code is available at https://github.com/kschwethelm/looped-lm-scaling. ## Predicate Neuron: Differentiable Quantization of Predicate Relations in Gradient-based Learning DevFeed: [Predicate Neuron: Differentiable Quantization of Predicate Relations in Gradient-based Learning]() Original publisher: [Read original article]() Author: Ioannis Konstantoulas, Iliana Loi, Kyriakos Sgarbas Published: 2026-10-30T04:00:00Z Content type: article Language: en Sources: [arXiv: Artificial Intelligence](), [arXiv: Machine Learning]() Topics: [Machine learning](), [Deep learning](), [Neural Network](), [Keras](), [PyTorch]() Tags: [backpropagation](), [benchmarks](), [convergence](), [cs-ai](), [cs-lg](), [deep-learning](), [deep-neural-networks](), [fully-connected](), [gradient](), [keras](), [learning](), [logic](), [logical](), [machine-learning](), [mechanism](), [module](), [native](), [networks](), [neural-networks](), [neuron](), [predicate](), [quantization]() ### AI overview The Native Predicate Neuron adds differentiable first-order logic reasoning to standard neural networks. A single trainable parameter selects between relational operations; a dual-nature gradient mechanism executes hard logical operations during training while preserving differentiability for backpropagation. Quantizing both inputs and the choice state is intended to help approximate continuous and discontinuous functions. The abstract reports evaluations in fully connected, convolutional, recurrent, and Transformer architectures on MNIST and Wikitext, with generally faster convergence, higher accuracy, and lower loss, and stronger gains on tasks involving monotonic reasoning or discontinuities. Implementations are available in Keras and PyTorch. These are abstract-level claims; specific benchmark results and limitations are not provided. ### Source excerpt arXiv:2610.37727v1 Announce Type: cross Abstract: In machine learning, symbolic reasoning approaches often rely on static architectures that hinder integration into deep neural networks and gradient-based optimization. To address this, we introduce the Native Predicate Neuron, a differentiable logic-reasoning neural module that enriches standard architectures with symbolic reasoning capabilities via first-order logic quantization. The proposed neuron models relational logic by learning to alternate between operations using a single trainable parameter. It features a tailored gradient mechanism with a dual-nature formulation: executing hard logical operations during training while maintaining differentiability for backpropagation. By quantizing both its input and choice state, the Predicate Neuron approximates continuous and non-continuous functions, effectively resolving discontinuities that standard deep learning architectures struggle with. We evaluate the Predicate Neuron across Fully-Connected, Convolutional, Recurrent, and Transformer architectures on benchmarks like MNIST and Wikitext. Results show that incorporating Predicate Neurons accelerates convergence, boosts accuracy, and lowers loss in general, with particularly strong improvements in tasks requiring monotonic reasoning or handling discontinuities. Overall, the Predicate Neuron natively integrates into standard neural networks to enable fuzzy logic operations and is available in Keras and PyTorch. ## LRMoE: Demand-Side Load Re-Calibration for Efficient MoE Training on Memory-Constrained Nodes DevFeed: [LRMoE: Demand-Side Load Re-Calibration for Efficient MoE Training on Memory-Constrained Nodes]() Original publisher: [Read original article]() Author: Jiaming Hu, Zhida Jiang, Zhaolong Xing, Yu Li, Zhen Chen, Ke Zhang Published: 2026-10-30T04:00:00Z Content type: article Language: en Sources: [arXiv: Artificial Intelligence](), [arXiv: Distributed Computing](), [arXiv: Machine Learning]() Topics: [Mixture of Experts (MoE)](), [Multi-GPU]() Tags: [algorithm](), [backpropagation](), [bottlenecks](), [buffer](), [capacity](), [communication](), [compaction](), [contribution](), [cs-ai](), [cs-dc](), [cs-lg](), [demand](), [execution](), [experiments](), [experts](), [gpu](), [layer](), [load](), [memory](), [metric](), [mixture-of-experts-moe](), [moe](), [training]() ### AI overview LRMoE reduces MoE training time under GPU memory constraints by recalibrating which token-expert pairs contribute during backpropagation. It plans expert placement and retention quotas during the forward pass while preserving all routed computations; during backpropagation, it estimates each pair's contribution, retains pairs within quota, and routes them among eligible copies of their original experts. Physical compaction then reduces communication payloads, computation, and execution-buffer demand. On Qwen3-30B-A3B and GLM4.5-106B-A12B, the authors report 21.2-44.8% less cumulative MoE-layer training time than SYMI to reach fixed-loss targets. The supplied evidence is an abstract, so it does not provide experimental details or broader comparisons. ### Source excerpt arXiv:2610.37720v1 Announce Type: cross Abstract: Sparse Mixture-of-Experts (MoE) models offer scalable capacity for Web applications, but uneven routing creates computation and communication bottlenecks. Existing placement and replication methods redistribute this workload, yet additional expert copies compete with token-related buffers for limited GPU memory. We present LRMoE, a memory-aware MoE training system that jointly recalibrates backward demand and balances the retained workload. During forward execution, it plans retention quotas and expert placement under resource constraints while preserving all routed computations. During backpropagation, a load recalibration metric estimates each token-expert pair's contribution to the expert update and selects pairs within these quotas. A capacity-aware routing algorithm distributes retained pairs among eligible instances of their original experts. Physical compaction then forms smaller communication payloads and variable-size grouped GEMMs, reducing backward traffic, computation, and execution-buffer demand. Experiments on Qwen3-30B-A3B and GLM4.5-106B-A12B show that LRMoE reduces cumulative MoE-layer training time to fixed-loss targets by 21.2-44.8% relative to SYMI. ## Follow the Gradient: Tracing Safety Degradation Back to Training Data DevFeed: [Follow the Gradient: Tracing Safety Degradation Back to Training Data]() Original publisher: [Read original article]() Author: Saket Reddy, Amro Abbas, Anshuman Suri, Ari Morcos, David Schwab, Pratyush Maini Published: 2026-10-28T04:00:00Z Content type: article Language: en Sources: [arXiv: Artificial Intelligence](), [arXiv: Machine Learning]() Topics: [Machine learning](), [Large Language Model](), [post-training](), [ai security](), [LLM security](), [hugging face]() Tags: [attack](), [backpropagation](), [behavior](), [behaviors](), [blocks](), [boosting](), [build](), [code](), [cs-ai](), [cs-cl](), [cs-lg](), [data](), [datasets](), [embedding](), [examples](), [execution](), [filter](), [fine-tuning](), [gradient](), [safety](), [tracing](), [training](), [training-data]() ### AI overview SimpleLoRIF uses gradient embeddings to identify benign-looking fine-tuning examples whose gradients align with unsafe behavior. Restricting backpropagation to the final four transformer blocks lets it process datasets with around a million examples. In an emergent-misalignment study, gradient attribution linked insecure-code examples to broader harmful behavior, with AUC up to 0.865. The researchers used the method to remove the 20% most aligned examples from Tulu 3, creating SafeTulu; across Qwen, Llama, and Olmo, training on it reduced GCG attack success by up to 22% versus the original, while largely preserving or sometimes improving capability. It outperformed LLM-as-a-judge and prior gradient-based filtering in the reported evaluation. Analysis of removed examples pointed to possible sources of safety degradation including tool-execution failures and narrative arithmetic. A Hugging Face Space allows researchers to customize filtering, such as its removal fraction or subsets. ### Source excerpt arXiv:2610.37481v1 Announce Type: cross Abstract: Fine-tuning Large Language Models (LLMs) on instruction data can quietly erode their safety, even when no training example looks harmful. Because this safety-degrading data looks benign, standard content filters struggle to catch it. We tackle this problem by developing SimpleLoRIF, a scalable gradient embedding method that identifies fine-tuning examples whose gradients align with unsafe behavior. By simplifying a low-rank attribution method and restricting backpropagation to the final four transformer blocks, SimpleLoRIF can embed datasets on the order of a million examples, boosting gradient attribution from a small-scale analysis tool into a practical filter for full post-training datasets. We first validate it on emergent misalignment, where we find that gradients can connect insecure-code training data to the broadly harmful behaviors that appear after fine-tuning, with AUC as high as 0.865 in the gradient space of misaligned models. We then use SimpleLoRIF to build SafeTulu, a version of Tulu 3 with the 20% of examples most gradient-aligned with unsafe responses removed. Training on SafeTulu reduces GCG attack success rate by up to 22% across Qwen, Llama, and Olmo, outperforming both LLM-as-a-judge and prior gradient-based filtering, while largely preserving or even improving capability. Finally, we analyzed the removed data and identify possibly under-studied sources of safety degradation, such as tool-execution failures and narrative arithmetic. We release SafeTulu along with a Hugging Face Space that lets researchers build their own customized versions of SafeTulu to fine-tune on, such as by changing the removal fraction or filtering only certain subsets. ## Persistence, Exposure, and Credit Routing in Linear Temporal Predictive Coding DevFeed: [Persistence, Exposure, and Credit Routing in Linear Temporal Predictive Coding]() Original publisher: [Read original article]() Author: Akira Tanase, Julian Gould, Jeffrey Seely Published: 2026-10-28T04:00:00Z Content type: article Language: en Sources: [arXiv: Machine Learning]() Topics: [Predictive AI]() Tags: [backpropagation](), [cellular](), [cs-it](), [cs-lg](), [inference](), [learning](), [linear](), [machine-learning](), [math-it](), [networks](), [operators](), [persistence](), [recurrent](), [routing](), [spatial](), [temporal](), [theory]() ### AI overview In linear temporal predictive coding, temporal credit assignment reflects a tradeoff between spatial displacement and temporal error, shaped by persistence operator P and exposure operator Q. Spatial inference yields the effective recurrent transition F=PA, while future credit is filtered by P*. Under non-amplifying invariant recurrence, modes that preserve influence over long delays expose that influence only weakly as temporal error. The abstract says recurrence can address this by routing between persistent and exposed modes. This overview is based on the supplied abstract and excerpt. ### Source excerpt arXiv:2610.36412v1 Announce Type: new Abstract: Temporal Predictive Coding (tPC) networks offer a layer-local alternative to backpropagation. For linear tPC, we show, by applying tools from cellular sheaf theory, that temporal credit assignment is a tug-of-war between spatial displacement and temporal errors, whose ratios are modulated by persistence and exposure operators $P$ and $Q$. Spatial inference therefore gives us the effective recurrent transition $F=PA$, whilst future credit is filtered by $P^\ast$. This creates a persistence--error tradeoff in spatial modes: under non-amplifying invariant recurrence, modes that preserve influence over long delays simultaneously fail to expose that influence strongly as temporal error. Recurrence can circumvent this via routing between persistent and exposed modes. ## Nonlinear Direct Feedback Alignment for Scalable Backpropagation-Free Training DevFeed: [Nonlinear Direct Feedback Alignment for Scalable Backpropagation-Free Training]() Original publisher: [Read original article]() Author: Wu Lin, Yan Fan, Chenxiang Ma, Yinglan Feng, Jia Wang, Ka-Chun Wong, Qiuzhen Lin Published: 2026-10-28T04:00:00Z Content type: article Language: en Sources: [arXiv: Machine Learning]() Topics: [Deep neural networks](), [Neural Network](), [Artificial Intelligence](), [cifar-10](), [ImageNet]() Tags: [algorithm](), [arxiv](), [backpropagation](), [cifar-10](), [code](), [cs-lg](), [deep-neural-networks](), [dfa](), [experiments](), [feedback](), [fixed](), [free](), [gap](), [hidden](), [layer](), [learning](), [lightweight](), [locking](), [matrices](), [network](), [training]() ### AI overview Nonlinear Direct Feedback Alignment (NDFA) aims to narrow the performance gap between direct feedback alignment (DFA) and backpropagation (BP). DFA sends output error directly to each hidden layer through fixed feedback matrices; the authors argue these matrices have limited ability to capture nonlinear mappings from hidden representations to the output. NDFA instead uses a lightweight nonlinear surrogate subnetwork to construct each layer's feedback and updates that subnetwork during training. The abstract reports that experiments with residual networks and vision transformers on CIFAR-10, CIFAR-100, SVHN, and ImageNet show NDFA outperforming existing DFA-based methods and substantially narrowing the gap to BP while retaining efficient parallel training. The supplied text does not include experimental details or quantitative results. ### Source excerpt arXiv:2610.36718v1 Announce Type: new Abstract: Backpropagation (BP) remains the predominant algorithm for training deep neural networks, but it is biologically implausible and suffers from the backward locking problem, constraining parallelization across layers. Direct Feedback Alignment~(DFA), which propagates the global output error directly to each hidden layer via fixed feedback matrices, has emerged as a promising alternative for addressing these limitations. However, a significant performance gap persists between DFA and BP, which we attribute to the limited ability of fixed feedback matrices to capture the nonlinear transformation from hidden representations to the network output. To address this limitation, we propose Nonlinear Direct Feedback Alignment (NDFA), which constructs each hidden-layer feedback using a lightweight nonlinear surrogate subnetwork. To enhance the effectiveness of the feedback throughout training, we further propose a learning strategy to update the surrogate subnetwork, thereby better approximating the underlying nonlinear transformation. Extensive experiments across residual networks and vision transformers on CIFAR-10, CIFAR-100, SVHN, and ImageNet datasets demonstrate that NDFA consistently outperforms existing DFA-based methods and substantially narrows the performance gap to BP, while maintaining efficient parallel training. The code is available at https://github.com/YanFan321/NDFA. ## Efficient Exact Unlearning for Federated Zeroth-Order Optimization DevFeed: [Efficient Exact Unlearning for Federated Zeroth-Order Optimization]() Original publisher: [Read original article]() Author: Weijun Yin, Yanqing Yao, Falko Dressler, Di Wang, Youming Tao Published: 2026-10-27T04:00:00Z Content type: article Language: en Sources: [arXiv: Distributed Computing](), [arXiv: Networking]() Topics: [Federated Learning](), [Algorithms]() Tags: [backpropagation](), [client](), [communication](), [convergence](), [cost](), [coupling](), [cs-ai](), [cs-dc](), [cs-lg](), [cs-ni](), [data](), [edge](), [experiments](), [exploits](), [federated](), [gaussian](), [guarantees](), [optimization](), [overlap](), [random](), [reflection](), [reuse](), [scalar](), [stable]() ### AI overview The paper presents two exact-unlearning methods for federated zeroth-order optimization, which avoids backpropagation and communicates random seeds and scalar coefficients. FedZOO-S uses client and mini-batch subsampling; FedZOUL-S preserves earlier updates and restarts at the first step affected by deletion. FedZOO-NS adds Gaussian noise to increase overlap between update distributions, and FedZOUL-NS uses rejection-reflection coupling to reuse affected updates when coupling is accepted. The authors say both methods produce a model and saved state jointly distributed as retraining on retained data, and establish stability, convergence, and expected unlearning-cost bounds. In experiments, they report up to 56.1% less cumulative unlearning time than re-optimization and communication overhead up to about four orders of magnitude lower than first-order baselines, with competitive model utility. ### Source excerpt arXiv:2610.35211v1 Announce Type: cross Abstract: Federated zeroth-order optimization (FedZOO) reduces the memory and communication costs of collaborative training for edge intelligence by avoiding backpropagation while communicating only random seeds and scalar coefficients. However, existing methods lack efficient mechanisms for exact unlearning to fully remove the influence of training data when users request deletion. A key insight is that total variation stability of the learning algorithm creates overlap between model distributions before and after data deletion, allowing exact unlearning to reuse computation through coupling. Based on this observation, we jointly design two stable zeroth-order learning algorithms and their corresponding exact-unlearning procedures: \textsc{FedZOO-S} obtains stability through client and mini-batch subsampling, enabling \textsc{FedZOUL-S} to perform efficient unlearning by preserving earlier updates and restarting from the first affected step. \textsc{FedZOO-NS} further combines subsampling with Gaussian noise to increase overlap between update distributions; \textsc{FedZOUL-NS} exploits this overlap through rejection--reflection coupling to reuse affected updates when the coupling is accepted. We prove exact unlearning for both algorithms: the model and saved state have the same joint distribution as retraining with the corresponding learner on retained data. We further establish stability and convergence guarantees for the learning algorithms and derive upper bounds on expected unlearning cost. Experiments demonstrate effective unlearning with competitive model utility and up to $56.1\%$ less cumulative unlearning time than re-optimization. Compared with first-order baselines, our methods reduce communication overhead by up to approximately four orders of magnitude. ## Neural LoFi with Backward Coupling: A Spectral Theory of Cross-Layer Feature Learning DevFeed: [Neural LoFi with Backward Coupling: A Spectral Theory of Cross-Layer Feature Learning]() Original publisher: [Read original article]() Author: Yatin Dandi, Matteo Vilucchio, Hugo Tabanelli, Florent Krzakala Published: 2026-10-26T04:00:00Z Content type: article Language: en Sources: [arXiv: Machine Learning]() Topics: [Neural Network](), [Machine learning](), [Deep neural networks](), [Deep learning]() Tags: [activation](), [algorithm](), [backpropagation](), [cifar-10](), [components](), [cond-mat-dis-nn](), [continuous](), [correlation](), [coupling](), [cross](), [cs-lg](), [experiments](), [feature](), [features](), [fixed](), [fully-connected](), [function](), [hidden](), [layer](), [learning](), [neural](), [stat-ml](), [theory]() ### AI overview The paper extends Neural Low-Degree Filtering to model how backpropagation lets later layers reshape earlier features. Its analysis derives a local coupling between a neuron's incoming weights and its corresponding next-layer column; in a controlled small-initialization regime, this amplifies singular modes pairing earlier features with upper-layer responses. The proposed algorithm uses upper-layer sensitivities as sample-dependent effective labels for earlier spectral steps and corrected features for upward transmission. In a hierarchical multi-index model, it recovers lower-layer directions missed by the original forward label-weighted statistic. Experiments on binary CIFAR-10 report higher accuracy than forward Neural LoFi for fully connected and convolutional networks at matched widths. ### Source excerpt arXiv:2610.32849v1 Announce Type: new Abstract: During joint training, backpropagation allows later layers to reshape earlier representations, a feedback mechanism absent from purely layer-wise learning. We extend Neural Low-Degree Filtering (Neural LoFi), a recently proposed layer-wise spectral surrogate for feature learning, to capture this interaction. Starting from the exact backpropagation gradients, we derive a local coupling between a neuron's incoming weights and its corresponding next-layer column. In a controlled small-initialization regime with smooth activations and fixed surroundings, these dynamics amplify singular modes of a cross-layer correlation operator. Each growing mode pairs an earlier feature with an upper-layer response through shared neuron coefficients. This motivates a backward-coupling algorithm in which upper-layer sensitivities generate sample-dependent effective labels for earlier spectral steps, while corrected features refine upward transmission. A function-space variational principle characterizes the selected feature-response pairs by maximizing the label correlation of their product under kernel-norm constraints. For three-layer networks, we derive self-consistent continuous-time dynamics for the filtered hidden-layer features, the activation kernels, and the network output under the proposed backward coupling. In a hierarchical multi-index model, we establish a feature-recovery separation: backward coupling recovers lower-layer directions invisible to the original forward label-weighted statistic. The linear specialization recovers residual-dependent singular-mode dynamics, while paired modes provide a dynamical mechanism for linked low-rank covariance components consistent with the rainbow-network picture. Experiments with fully connected and convolutional networks on binary CIFAR-10 show improved accuracy over forward Neural LoFi at matched widths. ## BrainRotViT: Transformer-ResNet Hybrid for Explainable Modeling of Brain Aging from 3D sMRI DevFeed: [BrainRotViT: Transformer-ResNet Hybrid for Explainable Modeling of Brain Aging from 3D sMRI]() Original publisher: [Read original article]() Author: Wasif Jalal, Md Nafiu Rahman, Atif Hasan Rahman, M. Sohel Rahman Published: 2026-10-26T04:00:00Z Content type: article Language: en Sources: [arXiv: Machine Learning]() Topics: [Transformers](), [Image](), [Data Science]() Tags: [2-5d](), [3d](), [aging](), [attention](), [autism](), [backpropagation](), [brain](), [cnn](), [context](), [cross](), [cs-cv](), [cs-lg](), [datasets](), [embeddings](), [error](), [hybrid](), [interpretability](), [lightweight](), [localization](), [matrix](), [modeling](), [transformer]() ### AI overview BrainRotViT estimates brain age from structural MRI by freezing a Vision Transformer pretrained on an age-sex classification task, then feeding its slice embeddings to a lightweight 2D residual CNN; sex also enters at the final prediction layer. On 5,108 samples from 11 datasets, validation MAE was 3.43 years, while four independent cohorts (948 samples total) had MAEs of 4.79-5.35 years. The abstract reports better performance than six retrained CNN and transformer baselines. After age-bias correction and covariate adjustment, brain-age gap was associated with Alzheimer's disease, mild cognitive impairment, and autism spectrum disorder. Attention, guided backpropagation, saliency checks, and AAL3 localization highlighted several regions, including occipital, cerebellar, motor, temporal, and medial superior frontal areas. These are abstract-level results; the supplied text does not provide further methodological detail. ### Source excerpt arXiv:2511.15188v3 Announce Type: replace-cross Abstract: Accurate brain-age estimation from structural MRI provides a valuable biomarker for studying healthy aging and neurodegenerative disease. We propose BrainRotViT (Brain ResNet over trained Vision Transformer), a two-stage, slice-based (2.5D) framework that approximates volumetric context while combining the strengths of CNNs and transformers. A Vision Transformer is pretrained on an auxiliary age-sex composite classification task and subsequently frozen. Its per-slice embeddings are stacked into a 160 x 768 feature matrix and processed as a pseudo-image by a lightweight 2D residual CNN regressor, with subject sex incorporated at the final prediction layer. Trained and validated on 5,108 samples from 11 multi-site datasets and tested on four independent held-out cohorts comprising 948 samples, BrainRotViT achieves a validation mean absolute error (MAE) of 3.43 years (Pearson r = 0.98, $R^2$ = 0.96, Spearman $\rho$ = 0.97) and cross-cohort MAEs of 4.79-5.35 years, outperforming retrained 3D-ResNet, SFCN, 3D-ViT, Global-Local Transformer, Triamese-ViT, and TSAN baselines. After age-bias correction and covariate adjustment, brain-age gap is significantly associated with Alzheimer's disease, mild cognitive impairment, and autism spectrum disorder. An interpretability pipeline combining ViT attention with guided backpropagation, saliency sanity checks, and AAL3-based anatomical localization identifies aging-related regions including the occipital lobe, cerebellum, precentral and postcentral gyri, temporal lobes, and medial superior frontal gyrus. BrainRotViT thus provides an efficient and interpretable bridge between CNN and transformer paradigms, demonstrating cross-cohort generalization while yielding biologically meaningful insights into brain aging. ## ENCL: Enabling Efficient Neuromorphic Continual Learning with Local Learning Rule and Sub-Network Freezing Strategy DevFeed: [ENCL: Enabling Efficient Neuromorphic Continual Learning with Local Learning Rule and Sub-Network Freezing Strategy]() Original publisher: [Read original article]() Author: Rachmad Vidya Wicaksana Putra, Shuochen Zhou, Hassan Raza, Muhammad Shafique Published: 2026-10-26T04:00:00Z Content type: article Language: en Sources: [arXiv: Artificial Intelligence](), [arXiv: Machine Learning]() Topics: [Machine learning](), [Neural Network](), [Artificial Intelligence](), [Model Development]() Tags: [backpropagation](), [cifar-10](), [continual-learning](), [cost](), [cs-ai](), [cs-lg](), [cs-ne](), [energy](), [lightweight](), [local](), [memory](), [neuromorphic](), [power-consumption](), [scalability](), [til]() ### AI overview ENCL aims to make neuromorphic continual learning more efficient without memory-heavy replay or costly global learning rules. It uses a temporally and spatially local learning rule for credit assignment and freezes task-specific subnetworks to avoid expanding the network; training also uses fewer epochs and adjusted learning rates and timesteps. In task-incremental experiments on Split MNIST, Split Fashion MNIST, Split CIFAR-10, and Split CIFAR-100, it reportedly achieved accuracy comparable to state-of-the-art replay methods while reducing memory cost by up to 88%, training time by up to 51%, average power by up to 40%, and energy by up to 65%. These are reported maximum reductions across the tested datasets. ### Source excerpt arXiv:2610.31927v1 Announce Type: cross Abstract: Continual learning (CL) in neural networks is a promising learning approach for updating the knowledge of deployed systems based on new data encountered at run time, that have not been learned during the offline training phase. To enable their power/energy-efficient CL processing, neuromorphic continual learning (NCL) concept with sparse event-based operations have recently been introduced. However, state-of-the-art NCL methods typically employ the memory-hungry data/experience replay, coupled with costly global learning rule (i.e., backpropagation-based surrogate gradient). To address these limitations, we propose ENCL, a novel methodology that enables efficient NCL by employing a lightweight local learning rule and sub-network freezing strategy. ENCL leverages the temporally- and spatially-local learning rule to perform efficient credit assignment for training new tasks, alleviating the expensive cost of global learning rule. ENCL also employs a sub-network freezing strategy based on the given tasks to avoid the needs for network architecture expansion, hence further improving the efficiency and scalability of the NCL training process. To efficiently support this, we train each task with a small number of epochs as well as adjusted learning rate and timestep. Experiment results on task-incremental learning (TIL) scenario show that, our ENCL consistently achieves strong performance across different datasets as compared to state-of-the-art works. Specifically, ENCL achieves competitive accuracy comparable to that of the state-of-the-art data/experience replay, while reducing memory cost by up to 88%, NCL training time by up to 51%, average power consumption by up to 40%, and energy consumption by up to 65% across different benchmarking datasets (i.e., Split MNIST, Split Fashion MNIST, Split CIFAR-10, and Split CIFAR-100). ## Autoregressive Physics-Informed Neural Operator Surrogates and Joint 4-D Seismic-Production Bayesian History Matching DevFeed: [Autoregressive Physics-Informed Neural Operator Surrogates and Joint 4-D Seismic-Production Bayesian History Matching]() Original publisher: [Read original article]() Author: Clement Etienam, David Blom, Hari Pattathil, Tsubasa Onishi, Harpreet Sethi, Lalson Vincent, Suraj Pawar, Kishan Ramesh Kumar, Pandu Devarakota, Faruk O. Alpak Published: 2026-10-26T04:00:00Z Content type: article Language: en Sources: [arXiv: Machine Learning]() Topics: [Neural Network](), [Artificial Intelligence]() Tags: [2](), [abstract](), [accelerated](), [adaptive](), [alpha](), [arxiv](), [autoregressive](), [backpropagation](), [bayesian](), [bias](), [black](), [but](), [cs-lg](), [d](), [history](), [matching](), [neural](), [operator](), [physics](), [physics-informed](), [production](), [stat-ml]() ### AI overview The framework makes Bayesian history matching for a 100,188-cell synthetic model feasible within one GPU-hour by replacing the forward simulator with a fast autoregressive Fourier neural operator surrogate. It trains the rollout with a black-oil residual physics penalty, truncated backpropagation, and noise injection, and derives 4-D seismic ΔVp through a closed-form Gassmann transform. Assimilation proceeds in two stages: aREKI fits production data, then ES-MDA assimilates production and seismic data jointly. To address rank deficiency and unconstrained directions in the high-dimensional inversion, permeability and porosity are represented in a lower-dimensional chart using either a truncated cosine transform or a variational auto-encoder decoder. On held-out data, the surrogate reports R² above 0.999 for pressure and saturation and a 0.579-second rollout, about 2,500 times faster than the forward model. The abstract also states theoretical bounds on rollout error, contraction under a small residual defect, and truncation bias; results are demonstrated on a synthetic model. ### Source excerpt arXiv:2610.31677v1 Announce Type: cross Abstract: We present a surrogate-accelerated Bayesian history-matching framework, demonstrated on a realistic synthetic model ($N=100{,}188$ cells). The forward map $\mathcal{G}$ is replaced by a Fourier neural operator $\mathcal{G}_\theta$ trained in a closed loop, $\hat{x}_{n+1}=\mathcal{G}_\theta(u,\hat{x}_n)$, with the discrete black-oil residual $R(x_{n+1},x_n)$ as a physics-informed neural operator (PINO) penalty, truncated backpropagation and noise injection. We assimilate production data and 4-D seismic using adaptive regularised ensemble Kalman inversion (aREKI), and synthesise $\Delta V_p$ inside the rollout via a closed-form Gassmann transform of the surrogate, under a two-stage schedule (aREKI on production, then an ensemble smoother with multiple data assimilation, ES-MDA, on both). To temper the ill-posedness, we invert $(\log K,\phi)$ not in $\mathbb{R}^{2N}$, where $N_e\ll 2N$ leaves a rank-deficient covariance and a null space the data never inform, but on a chart $u=\Psi(\tilde m)$, $\tilde m\in\mathbb{R}^{d}$, $d\ll 2N$, given by a Parseval-exact truncated cosine transform or a variational auto-encoder decoder. The ensemble then spans its own search space; the directions the data cannot constrain are cut to those the chart can express, and every member decodes to a geologically plausible field. For the rollout we prove an exposure bound $W_1\le\sum_j L^{\,n-1-j}\mathbb{E}e_j$, a physics-penalty contraction $\|D_x\mathcal{G}_\theta\|\le\alpha<1$ under a small residual defect, and an $\mathcal{O}(\rho^K)$ truncation bias. A Loewner-order inequality $\mathcal{I}_p\preceq\mathcal{I}_p+\mathcal{I}_s$ explains why joint assimilation is no less informative. On held-out data, $\mathcal{G}_\theta$ attains $R^2>0.999$ on pressure and saturation and rolls out in $0.579$ s, a $\sim2500\times$ speed-up, bringing an $N_e=190$-member, $N_a=10$-iteration joint inversion within one GPU-hour. ## TTT in VLA Models: Exploring Training with Human Demonstrations DevFeed: [TTT in VLA Models: Exploring Training with Human Demonstrations]() Original publisher: [Read original article]() Author: Qiangong Zhou, Jiafeng Liang, Yimin Dong, Yang He, Hongxiang Li Published: 2026-10-26T04:00:00Z Content type: article Language: en Sources: [arXiv: Artificial Intelligence]() Topics: [Artificial Intelligence]() Tags: [backpropagation](), [context](), [coverage](), [cs-ai](), [cs-cv](), [cs-ro](), [cue](), [cups](), [execution](), [experiments](), [gradient](), [horizon](), [implementation](), [information](), [memory](), [models](), [objectives](), [robot](), [scale](), [selection](), [sequences](), [training](), [vla]() ### AI overview The study integrates Test-Time Training (TTT), which stores context in fast weights, into InternVLA-A1 and Pi0.5 to test whether human demonstrations guide robot actions. It trains on paired demonstrations and robot trajectories, with memory transfer between phases and truncated backpropagation, and compares objectives, gradient coverage, and contrastive regularization across video and frame-based cues. At the tested scale, TTT and paired-data training did not produce consistent demonstration-based target selection. Shorter sequences, full trajectory gradient connectivity, and contrastive regularization showed no clear improvement; with single-frame cues, real-robot runs still favored cups at fixed positions. The abstract says implementation details and training configurations are documented for inspection and replication. ### Source excerpt arXiv:2610.32094v1 Announce Type: cross Abstract: Long-horizon robot tasks require retaining and using past information. We integrate Test-Time Training (TTT), which stores context in fast weights, into InternVLA-A1 and Pi0.5 to study action generation conditioned on human demonstrations through training on paired demonstrations and robot trajectories. We implement memory transfer between the two phases and truncated backpropagation, and examine training objectives, gradient coverage, and contrastive regularization across video, short-video, and single-frame cue tasks. In our experiments, adding TTT and training on paired data at the current scale did not lead to consistent target selection based on demonstrations. Under the tested settings, shorter sequences, full trajectory gradient connectivity, and contrastive regularization did not clearly improve responsiveness to demonstrations. When demonstrations were reduced to single-frame cues, real-robot execution still showed a preference for cups at fixed positions. We document the implementation, training configurations, and observations to support further inspection and replication. ## Neural Networks with Differentiable Belief-Function Layers DevFeed: [Neural Networks with Differentiable Belief-Function Layers]() Original publisher: [Read original article]() Author: Antonin Sulc Published: 2026-10-23T04:00:00Z Content type: article Language: en Sources: [arXiv: Artificial Intelligence]() Topics: [Deep neural networks](), [Neural Network](), [Artificial Intelligence]() Tags: [backpropagation](), [class](), [cs-ai](), [deep](), [distribution](), [gradient](), [layers](), [neural-networks](), [pipeline]() ### AI overview The paper builds a deep evidential network that combines valid Dempster-Shafer mass functions with Yager's rule. Unlike Dempster's rule, whose conflict normalizer creates a gradient term that diverges as evidence sources disagree, Yager's combination is bilinear and avoids that term while sending conflict to the ignorance mass m(Θ). Trained only on hard labels, the network's ignorance mass orders human disagreement in visual and linguistic settings. Dempster's rule ranks contested inputs similarly, but removes conflict through normalization; on the most contested CIFAR-10H images, Yager's vacuity is reported as 9x larger and interpretable as ignorance without recalibration. For out-of-distribution detection, the method is competitive with strong single-pass baselines; a higher-scoring clamped Dempster variant becomes gradient-unstable when unclamped. ### Source excerpt arXiv:2610.30173v1 Announce Type: new Abstract: Yager's 1987 rule of combination $\boxplus$ was proposed on epistemic grounds: under high conflict, mass should be redistributed to ignorance rather than renormalised onto the class simplex. We observe that backpropagation needs the same rule: Dempster's normaliser $(1{-}\kappa)^{-1}$, where $\kappa$ is the conflict mass, puts a $(1{-}\kappa)^{-2}$ term in the gradient that diverges exactly when evidence sources disagree, while Yager's $\boxplus$ is bilinear and has no such term. Building on this, we construct a deep evidential pipeline in which every mass in the chain from heads to output is a valid Dempster--Shafer mass function, combined by Yager's rule. The output mass on the ignorance set, $m(\Theta)$, serves as the network's ``I do not know'' signal. The network is fit to hard labels and never sees an annotator distribution, yet $m(\Theta)$ orders human disagreement in two modalities, visual and linguistic. Dempster's rises too and ranks contested inputs about as well, but its normaliser divides the conflict out: Yager's vacuity is $9\times$ larger on the most contested CIFAR-10H images, so only Yager's is readable as an amount of ignorance without recalibration. For out-of-distribution detection it is competitive with strong single-pass baselines, while the clamped Dempster variant that scores higher becomes gradient-unstable once the clamp is removed. ## A Novel Schur Decomposition-Based Weight Projection Method for Stable State-Space Neural-Network Architectures DevFeed: [A Novel Schur Decomposition-Based Weight Projection Method for Stable State-Space Neural-Network Architectures]() Original publisher: [Read original article]() Author: Sergio Vanegas, Lasse Lensu, Fredy Ruiz Published: 2026-10-22T04:00:00Z Content type: article Language: en Sources: [arXiv: Machine Learning]() Topics: [Machine learning](), [Neural Network]() Tags: [architectures](), [backpropagation](), [computational-complexity](), [convergence](), [cs-lg](), [cs-sy](), [data](), [datasets](), [eess-sy](), [experiments](), [layers](), [learning](), [linear](), [machine-learning](), [matrix](), [models](), [projection](), [robust](), [scheme](), [stable](), [state](), [static](), [synthetic]() ### AI overview The method enforces asymptotic stability in discrete-time state-space neural-network layers by projecting the quasi-triangular factor of the state matrix's real Schur decomposition onto its nearest stable counterpart. The projection is compatible with backpropagation; an alternative pre-factorized formulation is also proposed. On synthetic linear systems, the methods reportedly match state-of-the-art accuracy and convergence rates with a marginal computational cost increase. In stacked architectures with static nonlinearities on real-world datasets, the lower weight count reportedly helps convergence without sacrificing accuracy. The evidence provided is an abstract, so it does not specify datasets, benchmarks, or detailed limitations. ### Source excerpt arXiv:2605.14489v3 Announce Type: replace Abstract: Building black-box models for dynamical systems from data is a challenging problem in machine learning, especially when asymptotic stability guarantees are required. In this paper, we introduce a novel stability-ensuring and backpropagation-compatible projection scheme based on the Schur decomposition for the state matrix of linear discrete-time state-space layers, as well as an alternative pre-factorized formulation of the methodology. The proposed methods dynamically project the quasi-triangular factor of the state matrix's real Schur decomposition onto its nearest stable peer, ensuring stable dynamics with minimal overparameterization. Experiments on synthetic linear systems demonstrate that, despite a marginal increase in computational complexity, the method achieves accuracy and convergence rates comparable to those of state-of-the-art stable-system identification techniques. Furthermore, the lower weight count facilitates convergence during training without sacrificing accuracy in stacked neural-network architectures with static nonlinearities targeting real-world datasets. These results suggest that the Schur-based projection provides a numerically robust framework for identifying complex dynamics on par with the state of the art while satisfying strict asymptotic-stability requirements. ## Flow Matching Policy Optimization with Mirror Descent and Entropy Constraints DevFeed: [Flow Matching Policy Optimization with Mirror Descent and Entropy Constraints]() Original publisher: [Read original article]() Author: Ting Gao, Stavros Orfanoudakis, Nan Lin, Winnie Daamen, Serge Hoogendoorn, Elvin Isufi Published: 2026-10-22T04:00:00Z Content type: article Language: en Sources: [arXiv: Machine Learning]() Topics: [Reinforcement learning](), [World models]() Tags: [backpropagation](), [benchmarks](), [conditional](), [cs-lg](), [diffusion](), [entropy](), [exploration](), [field](), [flow-matching](), [multimodal](), [optimization](), [policy](), [reinforcement-learning](), [reward](), [sample](), [temperature](), [transformation](), [trust]() ### AI overview FMER uses ODE-based flow matching to optimize expressive RL policies while making entropy tractable and avoiding gradients through long denoising chains. It corrects the entropy objective for density distortion from the tanh mapping to bounded actions, uses an advantage-weighted conditional flow matching loss as a simulation-free surrogate for policy mirror descent, and adjusts mirror descent temperature using effective sample size to enforce a trust region. The abstract reports superior performance on sparse-reward FrankaKitchen and competitive results on dense-reward MuJoCo benchmarks; it provides no detailed metrics in the supplied text. ### Source excerpt arXiv:2603.17685v4 Announce Type: replace Abstract: Balancing policy expressiveness with the exploration-exploitation trade-off is a core challenge in online Reinforcement Learning (RL). While Stochastic Differential Equation (SDE)-based diffusion policies can represent complex, multimodal action distributions, they suffer from two critical limitations: their stochastic reverse processes render entropy intractable (necessitating heuristic exploration), and computing policy gradients through long denoising chains is expensive and unstable. In this work, we show that ODE-based flow matching inherently resolves these issues by enabling both simulation-free policy optimization and tractable entropy computation. Building on this, we introduce Flow Matching Policy Optimization with Mirror Descent and Entropy Constraints (FMER). Our framework exploits this insight in three ways. First, we derive an analytic entropy objective that corrects for the density distortion caused by the $\tanh$ transformation (mapping an unbounded latent space to bounded actions), thereby facilitating entropy-constrained policy optimization. Second, we theoretically establish that minimizing an advantage-weighted conditional flow matching loss acts as a simulation-free surrogate for policy mirror descent. This steers the velocity field toward high-value regions while entirely avoiding backpropagation through the ODE solver. Finally, we dynamically tune the mirror descent temperature based on the effective sample size to enforce a robust trust region during training. Empirical evaluations demonstrate that FMER achieves superior performance on the challenging sparse-reward FrankaKitchen environment, while maintaining competitive results across standard dense-reward MuJoCo benchmarks. Code is available at https://github.com/gtanaelle/FMER.git. [Next page]()