# Deep learning

Deep learning is a subset of machine learning driven by multilayered neural networks.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## DAWN: Robust Neural Blind Deconvolution via Alternating Wiener Guidance

DevFeed: [DAWN: Robust Neural Blind Deconvolution via Alternating Wiener Guidance](<https://devfeed.tech/articles/dawn-robust-neural-blind-deconvolution-via-alternating-wiener-guidance-145555.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.25621>)

Author: Yida Pan, Yixuan Jia, Chaoyan Huang, Qing Qu, Ismail Alkhouri

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Image](<https://devfeed.tech/topics/image.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Neural Network](<https://devfeed.tech/topics/neural-network.md>), [Deep learning](<https://devfeed.tech/topics/deep-learning.md>)

Tags: [computer-vision](<https://devfeed.tech/tags/computer-vision.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [dataset](<https://devfeed.tech/tags/dataset.md>), [diffusion](<https://devfeed.tech/tags/diffusion.md>), [dip](<https://devfeed.tech/tags/dip.md>), [gaussian](<https://devfeed.tech/tags/gaussian.md>), [image](<https://devfeed.tech/tags/image.md>), [kernel](<https://devfeed.tech/tags/kernel.md>), [microscopy](<https://devfeed.tech/tags/microscopy.md>), [network](<https://devfeed.tech/tags/network.md>), [neural](<https://devfeed.tech/tags/neural.md>), [noise](<https://devfeed.tech/tags/noise.md>), [photography](<https://devfeed.tech/tags/photography.md>), [robust](<https://devfeed.tech/tags/robust.md>), [training-data](<https://devfeed.tech/tags/training-data.md>)

### AI overview

DAWN is a training-data-free deep image prior framework for blind image deconvolution, designed to reduce noise overfitting and improve blur-kernel estimates. It alternates between feeding model-based estimates into the network and refining them with the network's prior, using autoencoding regularization on both image and kernel networks. On natural and scientific images with Gaussian and Poisson noise, the authors report better image and kernel quality than state-of-the-art methods, including up to 4.5 dB over the next-best baseline on the Levin dataset at σ = 0.1. Without training images, DAWN is also reported to remain highly competitive with a diffusion model trained on many in-distribution FFHQ images.

### Source excerpt

arXiv:2611.25621v1 Announce Type: cross Abstract: Blind image deconvolution (BID) is an ill-posed inverse problem that aims to jointly recover a sharp image and blur kernel from a single degraded observation. Under nontrivial and high noise, which is common in photography and scientific imaging such as microscopy, BID remains particularly challenging, as existing methods suffer from noise overfitting and inaccurate kernel estimation. Existing training-data-free learning methods for BID, such as those based on the deep image prior (DIP), often utilize external guidance by including model-based estimates as additional loss terms, which underexploits the information these estimates carry especially under nontrivial noise. To address these challenges, we propose DAWN, a training-data-free DIP framework that alternates between injecting model-based estimates directly into the model input and refining this imperfect estimate with the network's own prior. To provide more reliable kernel estimates for these injections, we apply autoencoding regularization to both the image and kernel networks. On natural and scientific images under varying levels of Gaussian and Poisson noise, DAWN outperforms state-of-the-art methods in both image and kernel quality (achieving up to +4.5 dB over the next best baseline on the Levin dataset at $\sigma = 0.1$). Without any training images, DAWN remains highly competitive with a state-of-the-art diffusion model on FFHQ, despite the latter being trained on a massive number of in-distribution images.

## DL101 The Representation Capability of Neural Network Models

DevFeed: [DL101 The Representation Capability of Neural Network Models](<https://devfeed.tech/articles/dl101-the-representation-capability-of-neural-network-models-144995.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.24980>)

Author: Fernando Berzal

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Deep learning](<https://devfeed.tech/topics/deep-learning.md>), [Deep neural networks](<https://devfeed.tech/topics/deep-neural-networks.md>), [Neural Network](<https://devfeed.tech/topics/neural-network.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>)

Tags: [approximation](<https://devfeed.tech/tags/approximation.md>), [continuous](<https://devfeed.tech/tags/continuous.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-ne](<https://devfeed.tech/tags/cs-ne.md>), [deep](<https://devfeed.tech/tags/deep.md>), [deep-learning](<https://devfeed.tech/tags/deep-learning.md>), [error](<https://devfeed.tech/tags/error.md>), [gradient-descent](<https://devfeed.tech/tags/gradient-descent.md>), [guarantees](<https://devfeed.tech/tags/guarantees.md>), [neural-network](<https://devfeed.tech/tags/neural-network.md>), [regions](<https://devfeed.tech/tags/regions.md>), [relu](<https://devfeed.tech/tags/relu.md>), [stat-ml](<https://devfeed.tech/tags/stat-ml.md>), [tangent](<https://devfeed.tech/tags/tangent.md>), [training](<https://devfeed.tech/tags/training.md>), [universal-approximation](<https://devfeed.tech/tags/universal-approximation.md>), [universality](<https://devfeed.tech/tags/universality.md>)

### AI overview

Universal approximation shows that shallow neural networks with non-polynomial activations can approximate any continuous function on compact sets, but it gives no guarantees about model size, successful training, or generalization. Barron's results provide quantitative approximation rates for functions with bounded Fourier norm. Montúfar's result shows that deep networks can form exponentially more linear regions as depth increases; the excerpt states that, for the same parameter count, they are more expressive than shallow networks. These results explain representational capacity, not whether training will achieve it.

### Source excerpt

arXiv:2611.24980v1 Announce Type: cross Abstract: Universal approximation theorems (Cybenko, Funahashi, Hornik & Leshno) provide the mathematical justification for why neural networks are powerful enough to model complex, nonlinear phenomena. Shallow networks can approximate any continuous function on compact sets. Leshno's theorem clarified that non-polynomial activations are necessary for universality, which explains why ReLU, logistic, hyperbolic tangent, and other common activations work (but pure polynomials do not). However, approximation theorems provide no efficiency, training, nor generalization guarantees. They do not indicate how many neurons are needed for a given accuracy, whether gradient descent will find the right parameters, or whether networks trained on finite data will generalize well. Barron provided quantitative error bounds for particular function classes (approximation rates for functions with bounded Fourier norm), whereas Mont\'ufar was the first to show that expressivity grows exponentially with depth (in terms of the number of linear regions). Deep networks are strictly more expressive than shallow ones with the same number of parameters. Depth is fundamentally more powerful than width, providing a rigorous mathematical explanation for the use of deep learning models.

## X Hedging: towards an eXplainable artificial intelligence option Hedging framework

DevFeed: [X Hedging: towards an eXplainable artificial intelligence option Hedging framework](<https://devfeed.tech/articles/x-hedging-towards-an-explainable-artificial-intelligence-option-hedging-framework-145112.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.25870>)

Author: Bj{\\o}rn Andr\\'e Aaslund, Johannes Berge, Ayoub Haida, Ying Ni, Rita Pimentel

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [AI Research](<https://devfeed.tech/topics/ai-research.md>), [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>), [Deep learning](<https://devfeed.tech/topics/deep-learning.md>), [Neural Network](<https://devfeed.tech/topics/neural-network.md>)

Tags: [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [data](<https://devfeed.tech/tags/data.md>), [data-driven](<https://devfeed.tech/tags/data-driven.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [efficiency](<https://devfeed.tech/tags/efficiency.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [explainability](<https://devfeed.tech/tags/explainability.md>), [framework](<https://devfeed.tech/tags/framework.md>), [lightgbm](<https://devfeed.tech/tags/lightgbm.md>), [math-oc](<https://devfeed.tech/tags/math-oc.md>), [models](<https://devfeed.tech/tags/models.md>), [neural-network](<https://devfeed.tech/tags/neural-network.md>), [neural-networks](<https://devfeed.tech/tags/neural-networks.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [option](<https://devfeed.tech/tags/option.md>), [performance](<https://devfeed.tech/tags/performance.md>), [policies](<https://devfeed.tech/tags/policies.md>), [random](<https://devfeed.tech/tags/random.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [robustness](<https://devfeed.tech/tags/robustness.md>), [x](<https://devfeed.tech/tags/x.md>)

### AI overview

X Hedging replaces the neural networks used in Deep Hedging with LightGBM tree ensembles to estimate dynamic hedging strategies. It sequentially trains the models and adds stochasticity through randomized training orders. In controlled experiments, it reportedly performs comparably to Deep Hedging while improving computational efficiency and user-centered explainability, with substantially greater robustness when training data are limited. The supplied abstract does not provide experiment details or quantify these gains.

### Source excerpt

arXiv:2611.25870v1 Announce Type: new Abstract: We propose X Hedging, an alternative to the Deep Hedging framework that addresses computational and explainability limitations of neural-network-based hedging strategies. Deep Hedging leverages neural networks to learn flexible hedging policies but relies on opaque models and typically requires large training datasets, which can complicate interpretation and limit its effectiveness in data-constrained settings. X Hedging takes a step toward addressing these limitations by replacing neural networks with tree ensemble methods, specifically LightGBM, to estimate hedging strategies. This design provides a computationally efficient alternative while preserving strong hedging performance and moving toward more user-centered explainability. To integrate LightGBM into a reinforcement-learning-style hedging framework, we introduce an optimization approach that sequentially trains tree-ensemble models while introducing stochasticity through random training orders. In controlled experiments, X Hedging achieves performance comparable to Deep Hedging while offering greater user-centered explainability, improved computational efficiency, and substantially higher robustness when training data are limited. The numerical results demonstrate that tree-ensemble methods provide an effective computational alternative to neural networks for data-driven dynamic hedging.

## Statistical, Machine Learning and Deep Learning Methods for Missing Covariates: An Empirical Evaluation with Applications to Alzheimer's Disease Research

DevFeed: [Statistical, Machine Learning and Deep Learning Methods for Missing Covariates: An Empirical Evaluation with Applications to Alzheimer's Disease Research](<https://devfeed.tech/articles/statistical-machine-learning-and-deep-learning-methods-for-missing-covariates-an-empirical-evaluation-with-applications-to-alzheimer-s-disease-research-145025.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.25219>)

Author: Yiqing Pan, Sharon X. Xie, Shujie Ma, Angela L. Jefferson, Panpan Zhang

Published: 2026-11-20T05:00:00Z

Content type: comparison

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Deep learning](<https://devfeed.tech/topics/deep-learning.md>), [VAE](<https://devfeed.tech/topics/vae.md>)

Tags: [adversarial](<https://devfeed.tech/tags/adversarial.md>), [aging](<https://devfeed.tech/tags/aging.md>), [analysis](<https://devfeed.tech/tags/analysis.md>), [application](<https://devfeed.tech/tags/application.md>), [applications](<https://devfeed.tech/tags/applications.md>), [autoencoders](<https://devfeed.tech/tags/autoencoders.md>), [bias](<https://devfeed.tech/tags/bias.md>), [biomarkers](<https://devfeed.tech/tags/biomarkers.md>), [biomedical](<https://devfeed.tech/tags/biomedical.md>), [case](<https://devfeed.tech/tags/case.md>), [challenges](<https://devfeed.tech/tags/challenges.md>), [confidence](<https://devfeed.tech/tags/confidence.md>), [confidence-interval](<https://devfeed.tech/tags/confidence-interval.md>), [coverage](<https://devfeed.tech/tags/coverage.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [data](<https://devfeed.tech/tags/data.md>), [deep-learning](<https://devfeed.tech/tags/deep-learning.md>), [disease](<https://devfeed.tech/tags/disease.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [research](<https://devfeed.tech/tags/research.md>), [stat-ml](<https://devfeed.tech/tags/stat-ml.md>)

### AI overview

The study evaluates eight methods for handling missing covariates in biomedical research, including complete-case analysis, multiple imputation, machine learning techniques, and deep learning frameworks such as multilayer perceptrons, variational autoencoders, and generative adversarial networks. Simulations compare estimation bias, efficiency, standard error accuracy, confidence interval coverage, and computational burden across missingness mechanisms and proportions. An application to the Vanderbilt Memory and Aging Project examines implications for regression analysis. The abstract says the study offers practical method-selection recommendations, but the supplied excerpt does not report which methods performed best.

### Source excerpt

arXiv:2611.25219v1 Announce Type: cross Abstract: Missing data are common in biomedical research, particularly when biomarkers are costly or require invasive procedures to obtain, such as cerebrospinal fluid measures in Alzheimer's disease. Although a wide range of missing data methods exists, from traditional statistical approaches to modern machine learning (ML) and deep learning (DL) algorithms, their comparative performance under different missingness mechanisms remains insufficiently understood. In this study, we provide an empirical evaluation of eight methods for handling missing covariates, including complete-case analysis, multiple imputation, ML-based techniques, and DL frameworks such as multilayer perceptrons, variational autoencoders, and generative adversarial networks. We summarize their conceptual foundations, practical implementation considerations, strengths, and limitations. Through extensive simulations, we assess estimation bias, efficiency, standard error accuracy, confidence interval coverage, and computational burden across different missingness mechanisms and proportions. An application to the Vanderbilt Memory and Aging Project further illustrates the implications of different methods for regression analysis in a real-world biomedical setting. Finally, we provide practical recommendations for method selection and highlight remaining challenges that warrant future investigation.

## Detecting AI-Generated Videos with Spiking Neural Networks

DevFeed: [Detecting AI-Generated Videos with Spiking Neural Networks](<https://devfeed.tech/articles/detecting-ai-generated-videos-with-spiking-neural-networks-145327.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2605.05895>)

Author: Minsuk Jang, Yujin Yang, Hee-Seon Kim, Minseok Son, Younghun Kim, Changick Kim

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Video Analytics](<https://devfeed.tech/topics/video-analytics.md>), [Deep learning](<https://devfeed.tech/topics/deep-learning.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [ai-generated-video](<https://devfeed.tech/tags/ai-generated-video.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [detection](<https://devfeed.tech/tags/detection.md>), [mast](<https://devfeed.tech/tags/mast.md>), [motion](<https://devfeed.tech/tags/motion.md>), [residual](<https://devfeed.tech/tags/residual.md>), [semantic](<https://devfeed.tech/tags/semantic.md>), [sequence](<https://devfeed.tech/tags/sequence.md>), [snn](<https://devfeed.tech/tags/snn.md>), [spiking-neural-networks](<https://devfeed.tech/tags/spiking-neural-networks.md>), [temporal](<https://devfeed.tech/tags/temporal.md>), [video](<https://devfeed.tech/tags/video.md>), [videos](<https://devfeed.tech/tags/videos.md>)

### AI overview

MAST detects AI-generated videos from temporal evidence, addressing the cross-generator performance drop reported for existing approaches. The authors identify smoother pixel-level frame residuals and more compact semantic-feature trajectories in generated clips. They also observe that SNNs fire mainly at object and motion boundaries on fake clips, consistent with sparse temporal artifacts concentrated at moments of change. MAST combines a spike-driven branch that processes multi-channel temporal residuals with a frozen semantic encoder. On GenVideo, it reports 93.14% mean accuracy across 10 unseen generators under strict cross-generator evaluation, matching or surpassing the strongest ANN-based detectors. The supplied abstract does not provide further implementation details or establish performance beyond this benchmark.

### Source excerpt

arXiv:2605.05895v3 Announce Type: replace-cross Abstract: Modern AI-generated videos are photorealistic at the single-frame level, leaving inter-frame dynamics as the main remaining axis for detection. Existing detectors typically handle this temporal evidence in three ways: feeding the full frame sequence to a generic temporal backbone, reducing one dominant temporal cue to fixed video-level descriptors, or comparing temporal features to real-video statistics through a detection metric. These strategies degrade sharply under cross-generator evaluation, where artifact type and timescale vary across generators. On caption-paired benchmark, GenVidBench, we identify two signatures that prior detectors do not jointly exploit: AI-generated videos exhibit smoother frame-to-frame temporal residuals at the pixel level, and more compact trajectories in the semantic feature space, indicating a temporal smoothness gap at both levels. We further observe that, when raw video is fed into a Spiking Neural Networks (SNNs), fake clips elicit firing predominantly at object and motion boundaries, unlike real clips, suggesting that the SNN responds to temporal artifacts localized at edges. These cues are sparse, asynchronous, and concentrated at moments of change, which makes SNNs a natural choice for this task: their event-driven, sparsely-activated dynamics align with the structure of the residual signal in a way that dense ANN backbones do not. Building on this observation, we propose MAST, a detector that processes multi-channel temporal residuals with a spike-driven temporal branch alongside a frozen semantic encoder for cross-generator generalization. On the GenVideo benchmark, MAST achieves 93.14\% mean accuracy across 10 unseen generators under strict cross-generator evaluation, matching or surpassing the strongest ANN-based detectors and demonstrating the practical applicability of SNNs to AI-generated video detection.

## Error Analysis of Deep Learning

DevFeed: [Error Analysis of Deep Learning](<https://devfeed.tech/articles/error-analysis-of-deep-learning-143375.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.23321>)

Author: Yuling Jiao, Peiying Wu, Jerry Zhijian Yang, Pingwen Zhang

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Deep learning](<https://devfeed.tech/topics/deep-learning.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>), [Representation learning](<https://devfeed.tech/topics/representation-learning.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>)

Tags: [approximation](<https://devfeed.tech/tags/approximation.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-na](<https://devfeed.tech/tags/cs-na.md>), [deep-learning](<https://devfeed.tech/tags/deep-learning.md>), [error](<https://devfeed.tech/tags/error.md>), [generative](<https://devfeed.tech/tags/generative.md>), [guarantees](<https://devfeed.tech/tags/guarantees.md>), [large-language-models](<https://devfeed.tech/tags/large-language-models.md>), [learning-theory](<https://devfeed.tech/tags/learning-theory.md>), [math-na](<https://devfeed.tech/tags/math-na.md>), [numerical-analysis](<https://devfeed.tech/tags/numerical-analysis.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [regression](<https://devfeed.tech/tags/regression.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [representation-learning](<https://devfeed.tech/tags/representation-learning.md>)

### AI overview

The review organizes deep learning guarantees around approximation, statistical, and optimization errors, tracing work from bounds on individual components to analyses that account for all three. It uses nonparametric regression and the Deep Ritz Method to illustrate the framework across different loss structures, then surveys applications in scientific machine learning, generative learning, reinforcement learning, representation learning, and large language models. Its focus is on identifying which errors are controlled, how they combine, and the assumptions required for guarantees about trained networks.

### Source excerpt

arXiv:2611.23321v1 Announce Type: cross Abstract: This review examines deep learning theory through the lens of error decomposition, systematically addressing approximation, statistical, and optimization errors. We trace the development of error analyses from bounds on individual error components to frameworks that account for all three. Using nonparametric regression and the Deep Ritz Method as dual running examples, we demonstrate how these analyses apply across fundamentally different loss structures. We then review representative analyses in scientific machine learning, generative learning, reinforcement learning, representation learning, and large language models, showing how learning errors affect solutions, distributions, and predictions through problem-specific arguments. Throughout, we clarify which error components are controlled, how they are combined, and what assumptions are needed to obtain guarantees for trained networks.

## Minimum Neural Weight Norm = Kolmogorov Complexity

DevFeed: [Minimum Neural Weight Norm = Kolmogorov Complexity](<https://devfeed.tech/articles/minimum-neural-weight-norm-kolmogorov-complexity-143259.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2605.10878>)

Author: Tiberiu Musat

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Neural Network](<https://devfeed.tech/topics/neural-network.md>), [compression and generalization](<https://devfeed.tech/topics/compression-and-generalization.md>), [Deep learning](<https://devfeed.tech/topics/deep-learning.md>), [Occam's razor machine learning](<https://devfeed.tech/topics/occam-s-razor-machine-learning.md>)

Tags: [cs-it](<https://devfeed.tech/tags/cs-it.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [kolmogorov-complexity](<https://devfeed.tech/tags/kolmogorov-complexity.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [math-it](<https://devfeed.tech/tags/math-it.md>), [neural-networks](<https://devfeed.tech/tags/neural-networks.md>), [sparsity](<https://devfeed.tech/tags/sparsity.md>), [theoretical](<https://devfeed.tech/tags/theoretical.md>), [weight](<https://devfeed.tech/tags/weight.md>)

### AI overview

For looped neural networks with constant-precision weights, the minimum weight norm needed to compute a function is asymptotically equivalent to that function's Kolmogorov complexity divided by its logarithm. The proof matches two bounds: sparse parameter patterns encode logarithmically many bits per nonzero parameter, while fixed-precision networks can be described by listing nonzero parameters with logarithmic addressing cost. The abstract says the bounds share leading constants, linking norm minimization with algorithmic simplicity as a way to study generalization.

### Source excerpt

arXiv:2605.10878v2 Announce Type: replace Abstract: Why do neural networks generalize better when their parameter norm is small? We prove that, in looped neural networks with constant-precision weights, the smallest weight norm of a network computing a function is asymptotically equivalent to the Kolmogorov complexity of that function, divided by its logarithm. The proof has two reductions: looped neural networks can encode information at a rate of logarithmically many bits per non-zero parameter encoded via the sparsity pattern, while any fixed-precision network can be described by enumerating its non-zero parameters with logarithmic addressing cost. Both bounds have the same leading constant factors, giving an asymptotic equivalence. More broadly, by establishing a theoretical link between norm minimization and algorithmic simplicity, we provide an avenue towards understanding generalization in modern deep learning.

## Decomposing SGD Dynamics in Neural Networks: Teacher-Induced Spikes and Variance Inflation

DevFeed: [Decomposing SGD Dynamics in Neural Networks: Teacher-Induced Spikes and Variance Inflation](<https://devfeed.tech/articles/decomposing-sgd-dynamics-in-neural-networks-teacher-induced-spikes-and-variance-inflation-143580.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.24908>)

Author: Samet Demir, Zafer Dogan

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Neural Network](<https://devfeed.tech/topics/neural-network.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Deep learning](<https://devfeed.tech/topics/deep-learning.md>), [machine learning overfitting](<https://devfeed.tech/topics/machine-learning-overfitting.md>)

Tags: [coupling](<https://devfeed.tech/tags/coupling.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [gradient-descent](<https://devfeed.tech/tags/gradient-descent.md>), [inflation](<https://devfeed.tech/tags/inflation.md>), [limit](<https://devfeed.tech/tags/limit.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [neural-networks](<https://devfeed.tech/tags/neural-networks.md>), [relu](<https://devfeed.tech/tags/relu.md>), [simulations](<https://devfeed.tech/tags/simulations.md>), [spikes](<https://devfeed.tech/tags/spikes.md>), [stat-ml](<https://devfeed.tech/tags/stat-ml.md>), [variance](<https://devfeed.tech/tags/variance.md>)

### AI overview

In two-layer networks trained by SGD on a multi-index teacher model, updates to first-layer directions orthogonal to the teacher begin isotropically but develop teacher-dependent spectral spikes. The authors identify this induced anisotropy as a mechanism for increased representation variance and worse generalization. Their analytically tractable Decomposed Dynamics surrogate has asymptotic test-error dynamics that coincide with SGD. In ReLU student simulations, freezing the orthogonal directions improves generalization compared with training them. The signal component also rapidly aligns with second-layer weights, indicating cross-layer coupling. These findings concern the analyzed setting and reported simulations.

### Source excerpt

arXiv:2611.24908v1 Announce Type: cross Abstract: We study two-layer neural networks trained by stochastic gradient descent (SGD) on a multi-index teacher model, focusing on how first-layer parameters orthogonal to the teacher subspace affect generalization. We decompose the first-layer weights into a signal component aligned with the teacher and an orthogonal complement, and analyze their joint SGD dynamics together with the second layer in the proportional limit. We show that although SGD updates to the complement are initially isotropic, the training dynamics induce teacher-dependent spectral spikes in the complement Gram matrix. This emergent anisotropy provides a concrete mechanism by which representation variance is amplified, and generalization degrades. To isolate this effect, we introduce Decomposed Dynamics, an analytically tractable surrogate whose asymptotic test-error dynamics coincide with those of SGD. In simulations with a ReLU student, freezing the directions of the orthogonal complement improves generalization relative to training them. Finally, we show that the signal component rapidly aligns with the second-layer weights, revealing strong cross-layer coupling even in this shallow setting. Together, our results provide a dynamical explanation for how feature learning outside the teacher subspace harms generalization.

## A Systematic Analysis of a Small Optimal Subspace for Physics-Informed Neural Networks: $n$-DAS $\\subset \\mathbb{R}^p$

DevFeed: [A Systematic Analysis of a Small Optimal Subspace for Physics-Informed Neural Networks: $n$-DAS $\\subset \\mathbb{R}^p$](<https://devfeed.tech/articles/a-systematic-analysis-of-a-small-optimal-subspace-for-physics-informed-neural-networks-n-das-subset-mathbb-r-p-143444.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.23773>)

Author: Ahmad AliAhmad, Khemraj Shukla

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Deep learning](<https://devfeed.tech/topics/deep-learning.md>), [Neural Network](<https://devfeed.tech/topics/neural-network.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>)

Tags: [convergence](<https://devfeed.tech/tags/convergence.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [math-ap](<https://devfeed.tech/tags/math-ap.md>), [math-oc](<https://devfeed.tech/tags/math-oc.md>), [neural-networks](<https://devfeed.tech/tags/neural-networks.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [physics-informed](<https://devfeed.tech/tags/physics-informed.md>), [subspace](<https://devfeed.tech/tags/subspace.md>)

### AI overview

The paper argues that training movement aligns more closely with gradient-aligned Hessian directions than with the directions of highest curvature. It defines the dominant aligned subspace (DAS) by selecting eigenvectors from a full Hessian eigendecomposition according to gradient alignment. In PINN benchmarks for Poisson, heat, wave, and viscous Burgers problems, DAS-restricted training with a line search matches full-space training, even when the projected Hessian remains ill-conditioned. The authors also extend the analysis to Gauss-Newton within DAS; the supplied text does not detail that analysis.

### Source excerpt

arXiv:2611.23773v1 Announce Type: new Abstract: The search for a low-dimensional subspace that governs neural network training has been a persistent goal in optimization. Yet subspaces defined by Hessian eigenvalue magnitude, such as the dominant/bulk decomposition underlying Dom-SGD and Bulk-SGD, have repeatedly failed to capture where optimization takes place, and adaptive Krylov-based alternatives built on energy-fraction thresholds show promise but still struggle at lower eigenmodes. We argue this failure is structural: the directions of highest curvature are not, in general, the directions along which training moves. We introduce the dominant aligned subspace (DAS), constructed from a full eigendecomposition of the Hessian but selected by gradient alignment rather than eigenvalue magnitude, as a tool for understanding optimization dynamics and a foundation for designing methods that exploit them. To expose this structure under the hardest conditions, we use physics-informed neural networks (PINNs) as a testbed: differential operators from the parabolic, elliptic, and hyperbolic classes induce severely ill-conditioned, anisotropic loss landscapes. Across Poisson, heat, wave, and viscous Burgers benchmarks DAS-restricted training with a line search matches full-space training even where the projected Hessian remains ill-conditioned, showing empirically that convergence does not require all Hessian eigendirections but only a low-dimensional, gradient-aligned subset. We extend the analysis to Gauss-Newton within the DAS and discuss what this structure implies for future subspace-aware optimizers.

## Biologically-Inspired Spike Encoding and Optimization for Spiking Neural Networks

DevFeed: [Biologically-Inspired Spike Encoding and Optimization for Spiking Neural Networks](<https://devfeed.tech/articles/biologically-inspired-spike-encoding-and-optimization-for-spiking-neural-networks-143367.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.23242>)

Author: Ayaan Qayyum, Bruce Yi Bu, Zicong Li, Jingfeng Zhou

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Deep learning](<https://devfeed.tech/topics/deep-learning.md>), [Neural Network](<https://devfeed.tech/topics/neural-network.md>)

Tags: [adaptive](<https://devfeed.tech/tags/adaptive.md>), [capacity](<https://devfeed.tech/tags/capacity.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [delta](<https://devfeed.tech/tags/delta.md>), [dog](<https://devfeed.tech/tags/dog.md>), [encoding](<https://devfeed.tech/tags/encoding.md>), [energy-efficiency](<https://devfeed.tech/tags/energy-efficiency.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [fashion](<https://devfeed.tech/tags/fashion.md>), [latency](<https://devfeed.tech/tags/latency.md>), [mnist](<https://devfeed.tech/tags/mnist.md>), [neural-networks](<https://devfeed.tech/tags/neural-networks.md>), [neuron](<https://devfeed.tech/tags/neuron.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [parameter](<https://devfeed.tech/tags/parameter.md>), [spiking-neural-networks](<https://devfeed.tech/tags/spiking-neural-networks.md>), [training](<https://devfeed.tech/tags/training.md>)

### AI overview

On MNIST and Fashion-MNIST, biologically motivated spike encoders improved accuracy most when model capacity was constrained, without increasing model size. The experiments compared rate, latency, DVS-style delta modulation, and Difference-of-Gaussians (DoG) encoding; DoG and delta modulation produced sparser spike trains than rate coding. K-FAC gave modest but consistent accuracy gains over Adam-trained baselines. Rouser's learnable per-neuron thresholds supported homeostatic firing-rate regulation and raised Fashion-MNIST accuracy from 87.37% to 88.13%. The evidence is limited to the abstract's reported experiments.

### Source excerpt

arXiv:2611.23242v1 Announce Type: new Abstract: Spiking Neural Networks (SNNs) offer theoretical energy-efficiency advantages over conventional deep networks, but realizing these advantages on vision tasks requires sparse spike representations that naive rate-coding methods fail to provide. We combine biologically motivated spike-encoding schemes with two training improvements: Rouser adaptive-threshold surrogate gradients and Kronecker-Factored Approximate Curvature (K-FAC) second-order optimization. The spike encoding schemes experimented with are rate coding, latency coding, DVS-style delta modulation, and Difference-of-Gaussians (DoG) filtering. Experiments on MNIST and Fashion-MNIST demonstrate that biologically inspired encoders yield the largest accuracy gains in capacity-constrained regimes and serve as an effective parameter-efficiency lever. DoG and delta modulation encoders produce significantly sparser spike trains than rate coding. K-FAC yields modest but consistent accuracy improvements over Adam-trained baselines, and Rouser's learnable per-neuron thresholds improve Fashion-MNIST accuracy from 87.37% to 88.13% by enabling homeostatic firing-rate regulation. Biologically grounded pre-processing methods can help close the accuracy gap between SNNs and non-spiking baselines without increasing model size.

## Optimal Deep Nonparametric Regression with Uniformly Bounded Lipschitz Constants

DevFeed: [Optimal Deep Nonparametric Regression with Uniformly Bounded Lipschitz Constants](<https://devfeed.tech/articles/optimal-deep-nonparametric-regression-with-uniformly-bounded-lipschitz-constants-143467.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.23979>)

Author: Jiaxin Shi, Ziyuan Chen, Jian Huang, Hansheng Wang

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Deep neural networks](<https://devfeed.tech/topics/deep-neural-networks.md>), [Deep learning](<https://devfeed.tech/topics/deep-learning.md>)

Tags: [adversarial](<https://devfeed.tech/tags/adversarial.md>), [approximation](<https://devfeed.tech/tags/approximation.md>), [bounded](<https://devfeed.tech/tags/bounded.md>), [constraint](<https://devfeed.tech/tags/constraint.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [deep](<https://devfeed.tech/tags/deep.md>), [estimator](<https://devfeed.tech/tags/estimator.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [math-st](<https://devfeed.tech/tags/math-st.md>), [prediction](<https://devfeed.tech/tags/prediction.md>), [regression](<https://devfeed.tech/tags/regression.md>), [robustness](<https://devfeed.tech/tags/robustness.md>), [stat-ml](<https://devfeed.tech/tags/stat-ml.md>), [stat-th](<https://devfeed.tech/tags/stat-th.md>)

### AI overview

The paper presents deep ReLU regression with controlled input sensitivity while retaining near-optimal statistical performance. Its split spectral norm product constraint supports arbitrary-accuracy approximation of sufficiently smooth Hölder functions as network depth grows, even with the norm-product bound fixed. A penalized least squares estimator then achieves the optimal nonparametric rate up to a logarithmic factor with a uniformly bounded Lipschitz constant. The authors also report optimal prediction rates for two-stage learning and robustness guarantees under adversarial perturbations. Numerical studies find controlled sensitivity and prediction accuracy comparable to classical deep network methods.

### Source excerpt

arXiv:2611.23979v1 Announce Type: cross Abstract: Stability under input perturbations is necessary for deep neural network estimators in many applications involving noisy covariates or estimated features. Weight norm constraints are widely used to control sensitivity to input perturbations by bounding the Lipschitz constant of neural networks. However, existing approximation results under fixed bounds on standard norm products are insufficient to establish optimal statistical rates. In this paper, we establish a sharp approximation theory for deep ReLU networks under a split spectral norm product constraint. This theoretical result shows that sufficiently smooth H\"{o}lder functions can be uniformly approximated to arbitrary accuracy as the network depth increases, with the norm-product bound held fixed. We then propose a penalized least squares estimator for deep nonparametric regression. The resulting estimator has a uniformly bounded Lipschitz constant while still achieving the optimal nonparametric rate up to a logarithmic factor. As applications, we establish optimal prediction rates for two-stage learning and provide robustness guarantees under adversarial input perturbations. Numerical studies show that the penalized estimator has well-controlled sensitivity to input perturbations and prediction accuracy comparable to that of classical deep network methods.

## Memorization capacity of deep ReLU neural networks characterized by width and depth

DevFeed: [Memorization capacity of deep ReLU neural networks characterized by width and depth](<https://devfeed.tech/articles/memorization-capacity-of-deep-relu-neural-networks-characterized-by-width-and-depth-141545.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2603.09589>)

Author: Xin Yang, Yunfei Yang

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Deep neural networks](<https://devfeed.tech/topics/deep-neural-networks.md>), [Neural Network](<https://devfeed.tech/topics/neural-network.md>), [Deep learning](<https://devfeed.tech/topics/deep-learning.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>)

Tags: [activation](<https://devfeed.tech/tags/activation.md>), [c](<https://devfeed.tech/tags/c.md>), [capacity](<https://devfeed.tech/tags/capacity.md>), [construction](<https://devfeed.tech/tags/construction.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-na](<https://devfeed.tech/tags/cs-na.md>), [data](<https://devfeed.tech/tags/data.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [deep](<https://devfeed.tech/tags/deep.md>), [delta](<https://devfeed.tech/tags/delta.md>), [lower-bounds](<https://devfeed.tech/tags/lower-bounds.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [math-na](<https://devfeed.tech/tags/math-na.md>), [minimal](<https://devfeed.tech/tags/minimal.md>), [networks](<https://devfeed.tech/tags/networks.md>), [neural-networks](<https://devfeed.tech/tags/neural-networks.md>), [relu](<https://devfeed.tech/tags/relu.md>), [size](<https://devfeed.tech/tags/size.md>)

### AI overview

The paper constructs ReLU networks with adjustable width W and depth L that can memorize any N points in the unit ball separated by distance δ, with labels from a finite set of size C. The construction satisfies W²L² log W = O(N(log(δ⁻¹) + log C)); the paper also establishes width-depth lower bounds that explicitly depend on δ and C. When log(δ⁻¹) is on the order of log N, suitable width and depth give W²L² on the order of N. The authors state this order is optimal when log C is on the order of log N, and optimal up to logarithmic factors when log C grows more slowly than log N.

### Source excerpt

arXiv:2603.09589v2 Announce Type: replace Abstract: This paper studies the memorization capacity of deep neural networks with ReLU activation. Specifically, we investigate the minimal size of such networks to memorize any $N$ data points in the unit ball with pairwise separation distance $\delta$ and labels from a finite set of size $C$. Most prior studies characterize the memorization capacity by the number of parameters or neurons. We extend these results by constructing ReLU networks with adjustable width $W$ and depth $L$ satisfying $W^2L^2\log W=\mathcal{O}(N(\log(\delta^{-1})+\log C))$ that can memorize any such dataset. We also establish width-depth lower bounds that depend explicitly on $\delta$ and $C$ for networks capable of memorizing all such datasets. When $\log(\delta^{-1})\asymp\log N$, our construction achieves $W^2L^2\asymp N$ for suitable choices of width and depth. This order is optimal if $\log C\asymp\log N$, and optimal up to logarithmic factors if $\log C=o(\log N)$.

## Sharp feature-learning transitions and Bayes-optimal neural scaling laws in extensive-width networks

DevFeed: [Sharp feature-learning transitions and Bayes-optimal neural scaling laws in extensive-width networks](<https://devfeed.tech/articles/sharp-feature-learning-transitions-and-bayes-optimal-neural-scaling-laws-in-extensive-width-networks-141557.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2605.10395>)

Author: Minh-Toan Nguyen, Jean Barbier

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Neural Network](<https://devfeed.tech/topics/neural-network.md>), [scaling laws](<https://devfeed.tech/topics/scaling-laws.md>), [Deep learning](<https://devfeed.tech/topics/deep-learning.md>)

Tags: [bayes](<https://devfeed.tech/tags/bayes.md>), [cond-mat-dis-nn](<https://devfeed.tech/tags/cond-mat-dis-nn.md>), [cond-mat-stat-mech](<https://devfeed.tech/tags/cond-mat-stat-mech.md>), [cs-it](<https://devfeed.tech/tags/cs-it.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [feature](<https://devfeed.tech/tags/feature.md>), [laws](<https://devfeed.tech/tags/laws.md>), [learning](<https://devfeed.tech/tags/learning.md>), [math-it](<https://devfeed.tech/tags/math-it.md>), [networks](<https://devfeed.tech/tags/networks.md>), [neural](<https://devfeed.tech/tags/neural.md>), [scaling](<https://devfeed.tech/tags/scaling.md>), [scaling-laws](<https://devfeed.tech/tags/scaling-laws.md>), [sharp](<https://devfeed.tech/tags/sharp.md>), [stat-ml](<https://devfeed.tech/tags/stat-ml.md>)

### AI overview

In a high-dimensional teacher-student setting, the study derives Bayes-optimal learning behavior for a one-hidden-layer teacher whose width grows linearly with input dimension. Its fixed-point equations predict sequential, discontinuous jumps in recoverability of hierarchical features as data increases. The resulting effective width--the number of learnable features at a given data budget--connects two error regimes: n^(1/(2β)-1) during feature learning and n^-1 during refinement, both summarized by ε^BO = Θ(k_c d/n), for hierarchy exponent β > 1/2. Empirically, a student trained with Adam near the effective width attains these scaling laws with a small algorithmic gap. The analysis relies on a heuristic leave-one-out argument validated numerically; the abstract does not provide further methodological details.

### Source excerpt

arXiv:2605.10395v2 Announce Type: replace-cross Abstract: We study the information-theoretic limits of learning a one-hidden-layer teacher network with hierarchical features from noisy queries, in the context of knowledge transfer to a smaller student model. We work in the high-dimensional regime where the teacher width $k$ scales linearly with the input dimension $d$ -- a setting that captures large-but-finite-width networks and has only recently become analytically tractable. Using a heuristic leave-one-out decoupling argument, validated numerically throughout, we derive asymptotically sharp characterizations of the Bayes-optimal generalization error and individual feature overlaps via a system of closed fixed-point equations. These equations reveal that feature learnability is governed by a sequence of sharp phase transitions: as data grows, teacher features become recoverable sequentially, each through a discontinuous jump in overlap. This sequential acquisition underlies a precise notion of \textit{effective width} $k_c$ -- the number of learnable features at a given data budget $n$ -- which unifies two distinct scaling regimes: a feature-learning regime in which the Bayes-optimal generalization error $\varepsilon^{\rm BO}$ scales as $ n^{1/(2\beta)-1}$, and a refinement regime in which it scales as $n^{-1}$, where $\beta>1/2$ is the exponent of the power-law feature hierarchy. Both laws collapse to the single relation $\varepsilon^{\rm BO}=\Theta(k_c d/n)$. We further show empirically that a student trained with \textsc{Adam} near the effective width $k_c$ achieves these optimal scaling laws (up to a small algorithmic gap), and provide an information-theoretic account of the associated scaling in model size.

## HE-LRM: Encrypted Deep Learning Recommendation Models using Fully Homomorphic Encryption

DevFeed: [HE-LRM: Encrypted Deep Learning Recommendation Models using Fully Homomorphic Encryption](<https://devfeed.tech/articles/he-lrm-encrypted-deep-learning-recommendation-models-using-fully-homomorphic-encryption-141438.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2506.18150>)

Author: Akshath Mahajan, Karthik Garimella, Austin Ebel, Gabrielle De Micheli, Brandon Reagen

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Cryptography and Security](<https://devfeed.tech/sources/arxiv-cryptography-and-security.md>)

Topics: [FHE](<https://devfeed.tech/topics/fhe.md>), [homomorphic encryption](<https://devfeed.tech/topics/homomorphic-encryption.md>), [Deep learning](<https://devfeed.tech/topics/deep-learning.md>), [Embeddings](<https://devfeed.tech/topics/embeddings.md>)

Tags: [batching](<https://devfeed.tech/tags/batching.md>), [client](<https://devfeed.tech/tags/client.md>), [criteo](<https://devfeed.tech/tags/criteo.md>), [cs-cr](<https://devfeed.tech/tags/cs-cr.md>), [deep-learning](<https://devfeed.tech/tags/deep-learning.md>), [embedding](<https://devfeed.tech/tags/embedding.md>), [embedding-tables](<https://devfeed.tech/tags/embedding-tables.md>), [encrypted](<https://devfeed.tech/tags/encrypted.md>), [fhe](<https://devfeed.tech/tags/fhe.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [gpu-memory](<https://devfeed.tech/tags/gpu-memory.md>), [high-performance](<https://devfeed.tech/tags/high-performance.md>), [homomorphic-encryption](<https://devfeed.tech/tags/homomorphic-encryption.md>), [inference](<https://devfeed.tech/tags/inference.md>), [latency](<https://devfeed.tech/tags/latency.md>), [lookup](<https://devfeed.tech/tags/lookup.md>), [models](<https://devfeed.tech/tags/models.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [orion](<https://devfeed.tech/tags/orion.md>), [recommendation](<https://devfeed.tech/tags/recommendation.md>), [throughput](<https://devfeed.tech/tags/throughput.md>)

### AI overview

HE-LRM presents an end-to-end fully homomorphic encryption system for private deep learning recommendation inference, including sparse embedding lookups. Its deterministic compositional embedding compression uses client-side digit decomposition to avoid costly homomorphic selector construction and consumes one multiplicative level. A block-diagonal packing strategy applies optimized BSGS transformations across embedding tables and outputs encrypted embeddings for downstream inference. An extended Orion GPU backend performs a single-inference lookup in 9 ms. On Criteo, complete encrypted inference takes 154-247 ms server-side on one GPU; GPU memory capacity limits batching and thus latency-bound throughput.

### Source excerpt

arXiv:2506.18150v5 Announce Type: replace Abstract: FHE inference systems primarily target models with dense inputs, while Deep Learning Recommendation Models (DLRMs) contain sparse categorical features that require lookups into embedding tables. Encrypted lookups are challenging because indexing is not natively supported and must be expressed using SIMD arithmetic. A direct one-hot formulation does not scale. Prior work addresses parts of this problem through embedding compression or compact encrypted queries. However, it does not address complete DLRM inference, leaving open how private lookup interacts with downstream computation and memory constraints. We present HE-LRM, the first end-to-end FHE system for private DLRM inference. We use deterministic compositional embedding compression with client-side digit decomposition, avoiding expensive homomorphic selector construction while consuming only one multiplicative level. We further develop a block-diagonal packing strategy that performs lookups across multiple embedding tables using optimized BSGS linear transformations and produces encrypted embeddings directly compatible with downstream inference. We extend Orion with a high-performance GPU backend and achieve 9 ms for single-inference embedding lookup. We also show that GPU memory capacity limits batching and therefore the latency-bound throughput in such systems. On Criteo, HE-LRM achieves complete encrypted inference with server-side latencies of 154-247 ms on a single GPU. We open-source HE-LRM at https://github.com/baahl-nyu/orion

## Graph neural networks and the energetic cavity method for combinatorial optimization

DevFeed: [Graph neural networks and the energetic cavity method for combinatorial optimization](<https://devfeed.tech/articles/graph-neural-networks-and-the-energetic-cavity-method-for-combinatorial-optimization-141604.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2609.07456>)

Author: Joe Bacchus George, George T. Cantwell

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Combinatorial optimization](<https://devfeed.tech/topics/combinatorial-optimization.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Deep learning](<https://devfeed.tech/topics/deep-learning.md>), [Neural Network](<https://devfeed.tech/topics/neural-network.md>)

Tags: [combinatorial-optimization](<https://devfeed.tech/tags/combinatorial-optimization.md>), [cond-mat-dis-nn](<https://devfeed.tech/tags/cond-mat-dis-nn.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [graph-neural-networks](<https://devfeed.tech/tags/graph-neural-networks.md>)

### AI overview

The study evaluates graph neural networks (GNNs) for approximating ground states of Ising models, which can represent many combinatorial optimization problems. Unmodified GNNs perform worse than two mean-field-based heuristics: a leading-eigenvector method and min-sum, also called the energetic cavity method. Incorporating these heuristics through small GNN modifications considerably improves performance and makes the approach stronger than other deep-learning methods. However, at the same computational cost, simulated annealing is generally as good. These findings are reported in the abstract; implementation details are not included in the supplied text.

### Source excerpt

arXiv:2609.07456v2 Announce Type: replace-cross Abstract: We study the use of graph neural networks (GNNs) for finding approximate ground states of Ising models. Efficiently finding these ground states is of broad significance because many combinatorial optimization problems can be formulated as an Ising model with the appropriate choice of couplings and fields. Exactly solving these problems is hard but there are many good heuristic methods. A lineage of these heuristics build from mean-field approximations: one approach uses the leading eigenvector of an appropriately defined matrix, another is the min-sum algorithm, also known as the energetic cavity method. Without modification, GNNs perform worse than both of these methods. We consider small modifications to the GNN to incorporate these heuristics and find that this considerably improves performance. While the modified approach is much stronger than other deep learning approaches, we still find that simulated annealing is generally as good for the same computational cost.

## BernBRS: Certified Backward Reachability for Deep Bernstein Neural Networks

DevFeed: [BernBRS: Certified Backward Reachability for Deep Bernstein Neural Networks](<https://devfeed.tech/articles/bernbrs-certified-backward-reachability-for-deep-bernstein-neural-networks-141731.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.22083>)

Author: Ibrahim Albool, Yasser Shoukry

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Formal verification](<https://devfeed.tech/topics/formal-verification.md>), [Deep learning](<https://devfeed.tech/topics/deep-learning.md>), [Deep neural networks](<https://devfeed.tech/topics/deep-neural-networks.md>), [Neural Network](<https://devfeed.tech/topics/neural-network.md>)

Tags: [activation-functions](<https://devfeed.tech/tags/activation-functions.md>), [approximation](<https://devfeed.tech/tags/approximation.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [neural-networks](<https://devfeed.tech/tags/neural-networks.md>), [reachability-analysis](<https://devfeed.tech/tags/reachability-analysis.md>), [safety](<https://devfeed.tech/tags/safety.md>), [scalability](<https://devfeed.tech/tags/scalability.md>), [verification](<https://devfeed.tech/tags/verification.md>)

### AI overview

BernBRS computes certified inner and outer approximations of the backward reachable set for DeepBern-Nets, which use Bernstein polynomial activations. It refines an input-space cover by analyzing activation patterns and splitting where an exact decomposition of the relaxation gap predicts the greatest error reduction, rather than relying on a single convex bound. The abstract reports scalability to complex architectures and evaluates the method on a linearized closed-loop Half-Cheetah controller and a distilled Bernstein student of a Transformer block; it also empirically examines how these model-level guarantees relate to the original Half-Cheetah and TinyBERT classifier. The supplied evidence does not give quantitative results.

### Source excerpt

arXiv:2611.22083v1 Announce Type: new Abstract: Ensuring the safety of complex neural network systems requires mathematically rigorous verification. While backward reachability analysis is a critical tool for identifying safe initial states, computing tight, formal bounds remains notoriously difficult to scale. Recent advancements have demonstrated that DeepBern-Nets neural networks utilizing Bernstein polynomial activation functions possess advantageous mathematical properties that enable exceptionally fast and tight forward reachability analysis. Building upon this efficiency, this paper introduces BernBRS, a highly accurate and scalable framework for the backward reachability analysis of DeepBern-Nets. BernBRS simultaneously computes certified inner and outer approximations of the Backward Reachable Set (BRS). Rather than relying on single convex bounding, our method intelligently refines a cover of the input space by analyzing activation patterns, splitting the input domain precisely where an exact decomposition of the relaxation gap predicts the most significant reduction in error. This targeted refinement enables high levels of scalability to highly complex and massive architectures that remain intractable for existing certified methods. We demonstrate the efficacy of our framework on high-dimensional models, including a linearized closed-loop approximation of the Half-Cheetah controller and a distilled Bernstein student of a Transformer block and we empirically study how these model-level guarantees relate to the original Half-Cheetah and TinyBERT classifier, respectively.

## Learning Greens Operators through Hierarchical Neural Networks Inspired by the Fast Multipole Method

DevFeed: [Learning Greens Operators through Hierarchical Neural Networks Inspired by the Fast Multipole Method](<https://devfeed.tech/articles/learning-greens-operators-through-hierarchical-neural-networks-inspired-by-the-fast-multipole-method-141529.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2509.20591>)

Author: Emilio McAllister Fognini, Marta M. Betcke, Ben T. Cox

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Neural Network](<https://devfeed.tech/topics/neural-network.md>), [Deep learning](<https://devfeed.tech/topics/deep-learning.md>)

Tags: [algorithm](<https://devfeed.tech/tags/algorithm.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [fast](<https://devfeed.tech/tags/fast.md>), [learning](<https://devfeed.tech/tags/learning.md>), [neural](<https://devfeed.tech/tags/neural.md>), [neural-networks](<https://devfeed.tech/tags/neural-networks.md>), [operators](<https://devfeed.tech/tags/operators.md>)

### AI overview

Neural FMM applies the Fast Multipole Method's hierarchical information flow to learn the Green's operator of an elliptic PDE. It separates local and far-field interactions and learns representations for each; the supplied abstract does not report evaluation results or performance comparisons.

### Source excerpt

arXiv:2509.20591v2 Announce Type: replace Abstract: The Fast Multipole Method (FMM) is an efficient numerical algorithm for computation of long-ranged forces in $N$-body problems within gravitational and electrostatic fields. This method utilizes multipole expansions of the Green's function inherent to the underlying dynamical systems. Despite its widespread application in physics and engineering, the integration of FMM with modern machine learning architectures remains underexplored. In this work, we propose a novel neural network architecture, the Neural FMM, that integrates the information flow of the FMM into a hierarchical machine learning framework for learning the Green's operator of an Elliptic PDE. Our Neural FMM architecture leverages a hierarchical computation flow of the FMM method to split up the local and far-field interactions and efficiently learn their respective representations.

## Gradient Boosting vs. Random Forest vs. Neural Networks: A 2026 Performance Benchmark

DevFeed: [Gradient Boosting vs. Random Forest vs. Neural Networks: A 2026 Performance Benchmark](<https://devfeed.tech/articles/gradient-boosting-vs-random-forest-vs-neural-networks-a-2026-performance-benchmark-137767.md>)

Original publisher: [Read original article](<https://machinelearningmastery.com/gradient-boosting-vs-random-forest-vs-neural-networks-a-2026-performance-benchmark/>)

Author: Vinod Chugani

Published: 2026-11-17T11:00:00Z

Content type: comparison

Language: en

Sources: [MachineLearningMastery.com](<https://devfeed.tech/sources/machinelearningmastery-com.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Benchmark](<https://devfeed.tech/topics/benchmark.md>), [Neural Network](<https://devfeed.tech/topics/neural-network.md>), [AI Foundation Models](<https://devfeed.tech/topics/ai-foundation-models.md>), [Deep learning](<https://devfeed.tech/topics/deep-learning.md>), [Hyperparameter optimization](<https://devfeed.tech/topics/hyperparameter-optimization.md>), [Transformer](<https://devfeed.tech/topics/transformer.md>), [catboost](<https://devfeed.tech/topics/catboost.md>), [mlp](<https://devfeed.tech/topics/mlp.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>)

Tags: [2026](<https://devfeed.tech/tags/2026.md>), [algorithm](<https://devfeed.tech/tags/algorithm.md>), [algorithms](<https://devfeed.tech/tags/algorithms.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [audio](<https://devfeed.tech/tags/audio.md>), [backpropagation](<https://devfeed.tech/tags/backpropagation.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [catboost](<https://devfeed.tech/tags/catboost.md>), [classification](<https://devfeed.tech/tags/classification.md>), [cnns](<https://devfeed.tech/tags/cnns.md>), [compare](<https://devfeed.tech/tags/compare.md>), [compute](<https://devfeed.tech/tags/compute.md>), [context-window](<https://devfeed.tech/tags/context-window.md>), [cpu](<https://devfeed.tech/tags/cpu.md>), [foundation-models](<https://devfeed.tech/tags/foundation-models.md>), [gradient-boosting](<https://devfeed.tech/tags/gradient-boosting.md>), [neural-networks](<https://devfeed.tech/tags/neural-networks.md>), [performance](<https://devfeed.tech/tags/performance.md>), [practical-machine-learning](<https://devfeed.tech/tags/practical-machine-learning.md>), [requirements](<https://devfeed.tech/tags/requirements.md>), [speed](<https://devfeed.tech/tags/speed.md>)

### AI overview

The excerpt describes two regimes for tabular data: tabular foundation models (TFMs) lead TabArena accuracy results on small-to-medium datasets, while gradient boosting remains stronger for large-scale data and regression. TabPFN-3.5 is reported to support up to 1 million rows and 20,000 features, but TFM limits vary and inference requires GPU memory to hold data in context. XGBoost, LightGBM, and CatBoost offer CPU-based training and low-latency inference, making them practical under compute or latency constraints. Random forest is a forgiving, quickly tuned baseline, especially with noisy data or limited tuning resources. Specialized networks have narrower advantages: TabR can outperform XGBoost on some classification tasks, while TabM and RealMLP reach parity with boosted trees on large datasets when carefully tuned and ensembled. For images, text, and audio, transformer architectures lead.

### Source excerpt

In this article, you will learn how gradient boosting, random forest, neural networks, and tabular foundation models compare on accuracy, speed, and data requirements in...

## Symbolic Branch Networks: Tree-Inherited Neural Models for Interpretable Multiclass Classification

DevFeed: [Symbolic Branch Networks: Tree-Inherited Neural Models for Interpretable Multiclass Classification](<https://devfeed.tech/articles/symbolic-branch-networks-tree-inherited-neural-models-for-interpretable-multiclass-classification-138719.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2507.01781>)

Author: Dalia N\\"oth-Rodr\\'iguez

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Neural Network](<https://devfeed.tech/topics/neural-network.md>), [Deep learning](<https://devfeed.tech/topics/deep-learning.md>), [model architecture](<https://devfeed.tech/topics/model-architecture.md>), [Model Development](<https://devfeed.tech/topics/model-development.md>), [Predictive AI](<https://devfeed.tech/topics/predictive-ai.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [analysis](<https://devfeed.tech/tags/analysis.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [based](<https://devfeed.tech/tags/based.md>), [basis](<https://devfeed.tech/tags/basis.md>), [branch](<https://devfeed.tech/tags/branch.md>), [but](<https://devfeed.tech/tags/but.md>), [can](<https://devfeed.tech/tags/can.md>), [class](<https://devfeed.tech/tags/class.md>), [classification](<https://devfeed.tech/tags/classification.md>), [components](<https://devfeed.tech/tags/components.md>), [configuration](<https://devfeed.tech/tags/configuration.md>), [connectivity](<https://devfeed.tech/tags/connectivity.md>), [contributions](<https://devfeed.tech/tags/contributions.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [dataset](<https://devfeed.tech/tags/dataset.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [models](<https://devfeed.tech/tags/models.md>), [networks](<https://devfeed.tech/tags/networks.md>), [neural](<https://devfeed.tech/tags/neural.md>)

### AI overview

Symbolic Branch Networks (SBNs) map each decision-tree root-to-parent-of-leaf path to a hidden neuron, linking teacher branches to neural components for feature- and branch-level interpretation. In the studied SBN1,0 configuration, the teacher-derived feature-to-branch matrix W1 is trained by gradient descent while the branch-to-class matrix W2 remains fixed. Across 29 multiclass tabular datasets, SBN1,0 beats its teacher ensemble on 28, raising mean accuracy from 84.0% to 87.9%; XGBoost and TabPFN average 87.3% and 91.3% on the same datasets. The teacher comparison is statistically significant (one-sided Wilcoxon signed-rank p=4.71x10⁻⁷). Predictions can be traced through selected branch neurons, their features, and class contributions. Across ten independently constructed teacher ensembles, one selected instance had consistent predictions but only partly stable local feature rankings.

### Source excerpt

arXiv:2507.01781v3 Announce Type: replace Abstract: Symbolic Branch Networks (SBNs) are neural models whose architecture is derived directly from an ensemble of decision trees. Each root-to-parent-of-leaf decision path is mapped to a hidden neuron, while the matrices W1 (feature-to-branch) and W2 (branch-to-class) encode the structural relationships induced by the teacher ensemble. This provides an explicit correspondence between teacher branches and neural components, enabling feature-level and branch-level interpretation while retaining the flexibility of gradient-based optimization. The primary configuration studied in this work trains the teacher-derived feature-to-branch matrix W1 through gradient-based learning while keeping the branch-to-class matrix W2 fixed; this configuration is denoted SBN1,0. Across 29 multiclass tabular datasets, this configuration improves upon the teacher ensemble on 28 of 29 datasets, increasing mean accuracy from 84.0% to 87.9%. For reference, XGBoost and TabPFN achieve mean accuracies of 87.3% and 91.3%, respectively, on the same datasets. A one-sided Wilcoxon signed-rank test on dataset-level mean accuracies indicates a statistically significant improvement of SBN1,0 over the teacher ensemble (p=4.71x10-7). Beyond predictive performance, SBN provides an explicit structural basis for interpretation. Global feature importance can be derived from the teacher-induced feature-to-branch connectivity, while individual predictions can be traced to selected branch-neurons, their associated features, and their class contributions. An analysis across ten independently constructed teacher ensembles shows consistent predictions for a selected instance but only partial stability in its local feature rankings.

## HELENA for 5G NR LEO NTN Channel Estimation: A Comparative Evaluation

DevFeed: [HELENA for 5G NR LEO NTN Channel Estimation: A Comparative Evaluation](<https://devfeed.tech/articles/helena-for-5g-nr-leo-ntn-channel-estimation-a-comparative-evaluation-137016.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2609.14735>)

Author: Miguel Camelo Botero, Nina Slamnik-Krije\\v{s}torac, Johann Marquez-Barja

Published: 2026-11-17T05:00:00Z

Content type: comparison

Language: en

Sources: [arXiv: Networking](<https://devfeed.tech/sources/arxiv-networking.md>)

Topics: [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Deep learning](<https://devfeed.tech/topics/deep-learning.md>), [Model Development](<https://devfeed.tech/topics/model-development.md>), [TensorRT](<https://devfeed.tech/topics/tensorrt.md>), [Jetson Orin](<https://devfeed.tech/topics/jetson-orin.md>), [RTX PRO](<https://devfeed.tech/topics/rtx-pro.md>)

Tags: [5g](<https://devfeed.tech/tags/5g.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [attention](<https://devfeed.tech/tags/attention.md>), [channel](<https://devfeed.tech/tags/channel.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-ni](<https://devfeed.tech/tags/cs-ni.md>), [cs-pf](<https://devfeed.tech/tags/cs-pf.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [deep-learning](<https://devfeed.tech/tags/deep-learning.md>), [doppler](<https://devfeed.tech/tags/doppler.md>), [eess-sp](<https://devfeed.tech/tags/eess-sp.md>), [embedded](<https://devfeed.tech/tags/embedded.md>), [energy](<https://devfeed.tech/tags/energy.md>), [estimator](<https://devfeed.tech/tags/estimator.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [fp16](<https://devfeed.tech/tags/fp16.md>), [high-performance](<https://devfeed.tech/tags/high-performance.md>), [inference](<https://devfeed.tech/tags/inference.md>), [jetson](<https://devfeed.tech/tags/jetson.md>), [latency](<https://devfeed.tech/tags/latency.md>), [leo](<https://devfeed.tech/tags/leo.md>), [low-earth-orbit](<https://devfeed.tech/tags/low-earth-orbit.md>), [non-terrestrial-networks](<https://devfeed.tech/tags/non-terrestrial-networks.md>), [ntn](<https://devfeed.tech/tags/ntn.md>)

### AI overview

The study finds HELENA's unchanged dual-attention architecture remains competitive for the evaluated LEO NTN channel-estimation task after retraining, without an NTN-specific redesign. Its SNR-averaged NMSE is within 0.29 dB of AttRNet with receiver compensation and 0.72 dB with residual impairments. HELENA beats global LMMSE in the compensated condition and all evaluated LMMSE variants under residual impairments, though profile-aware filtering is more accurate with compensation. FP16 TensorRT inference on an RTX PRO 4500 reaches 0.0573 ms P99, under the 0.5 ms budget. On a 10 W Jetson Orin NX, HELENA uses 66.0% less energy per sample than AttRNet, but its 0.9620 ms P99 misses that budget; only the less accurate LSiDNN-48 meets it. Results suggest embedded latency remains a challenge for accurate estimation.

### Source excerpt

arXiv:2609.14735v2 Announce Type: replace-cross Abstract: Deep Learning (DL)-based channel estimation has shown high accuracy and low latency in terrestrial 5G NR, but Low Earth Orbit (LEO) Non-Terrestrial Networks (NTNs) introduce Doppler and synchronization impairments that may require NTN-specific architectures. We investigate whether HELENA, a dual-attention neural channel estimator originally designed for terrestrial channels, remains effective after NTN retraining and suitable across high-performance and power-constrained inference platforms. Its unchanged architecture is trained on paired receiver-compensated and residual-impaired datasets and evaluated alongside terrestrial-origin, NTN-specific, and classical estimators. HELENA's SNR-averaged Normalized Mean Squared Error (NMSE), computed by arithmetic averaging in linear scale, remains within 0.29 and 0.72 dB of the most accurate learned estimator, AttRNet, in the two conditions, respectively. It outperforms global Linear Minimum Mean Squared Error (LMMSE) under compensation and all evaluated LMMSE variants under residual impairments, although profile-aware filtering achieves lower error under compensation. On an RTX PRO 4500, FP16 TensorRT inference reaches 0.0573 ms 99th-Percentile (P99) latency, below the adopted 0.5 ms estimator budget. On a 10 W Jetson Orin NX, HELENA consumes 66.0% less energy per sample than AttRNet, but its 0.9620 ms P99 exceeds the budget; only the less accurate LSiDNN-48 meets it. Thus, HELENA remains effective without NTN-specific redesign for the evaluated task, while accurate embedded inference remains a latency challenge.

## Neuro-Bayesian-Symbolic Residual Attention Shallow Network: Explainable Deep Learning for Cybersecurity Risk Assessment

DevFeed: [Neuro-Bayesian-Symbolic Residual Attention Shallow Network: Explainable Deep Learning for Cybersecurity Risk Assessment](<https://devfeed.tech/articles/neuro-bayesian-symbolic-residual-attention-shallow-network-explainable-deep-learning-for-cybersecurity-risk-assessment-137262.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2606.30953>)

Author: Nicolaie Popescu-Bodorin, Madeleine Togher

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Neural Network](<https://devfeed.tech/topics/neural-network.md>), [model architecture](<https://devfeed.tech/topics/model-architecture.md>), [Deep learning](<https://devfeed.tech/topics/deep-learning.md>), [Deep neural networks](<https://devfeed.tech/topics/deep-neural-networks.md>)

Tags: [adjustment](<https://devfeed.tech/tags/adjustment.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [attention](<https://devfeed.tech/tags/attention.md>), [bayesian](<https://devfeed.tech/tags/bayesian.md>), [blast-radius](<https://devfeed.tech/tags/blast-radius.md>), [bootstrap](<https://devfeed.tech/tags/bootstrap.md>), [causality](<https://devfeed.tech/tags/causality.md>), [cisa](<https://devfeed.tech/tags/cisa.md>), [cisa-kev](<https://devfeed.tech/tags/cisa-kev.md>), [completeness](<https://devfeed.tech/tags/completeness.md>), [confidence](<https://devfeed.tech/tags/confidence.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cve](<https://devfeed.tech/tags/cve.md>), [cybersecurity](<https://devfeed.tech/tags/cybersecurity.md>), [cybersecurity-risk-assessment](<https://devfeed.tech/tags/cybersecurity-risk-assessment.md>), [deep-learning](<https://devfeed.tech/tags/deep-learning.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [explainability](<https://devfeed.tech/tags/explainability.md>), [network](<https://devfeed.tech/tags/network.md>), [risk](<https://devfeed.tech/tags/risk.md>), [risk-assessment](<https://devfeed.tech/tags/risk-assessment.md>)

### AI overview

NBS-RASN proposes explainable cybersecurity risk assessment through an 80-neuron, 12-layer shallow network. A gatekeeper layer enforces five stated axioms before propagation; residual attention and feedback loops support context-dependent patterns. Risk scores combine a deterministic weighted component with expert adjustments traceable to named amplifiers, including blast radius and default exposure. The abstract reports validation on 20 open-source projects across OWASP Top 10:2025 categories and language risk classes, with confidence scores of 0.79-0.97. These are bootstrap, precision-recall-derived estimates against confirmed CVE incidents in CISA KEV and NVD, not softmax probabilities. The excerpt does not provide comparative results or further validation details.

### Source excerpt

arXiv:2606.30953v2 Announce Type: replace Abstract: We introduce NBS-RASN, a novel hybrid neural architecture for explainable cybersecurity risk evaluation in open-source software ecosystems. Unlike conventional deep models that trade interpretability for predictive power, it uses a meticulously designed shallow network that encodes domain knowledge, causal reasoning, and expert judgment as differentiable components. The network has 80 interpretable neurons across 12 layers, including a gatekeeper layer enforcing five epistemological axioms-precision, causality, falsifiability, transparency, and completeness-as hard constraints before propagation. Despite limited depth, non-linear residual attention and feedback loops give it deep-learning characteristics, enabling complex context-dependent risk patterns without becoming a black box. It produces fully decomposable risk scores: a deterministic weighted component plus an expert adjustment component, with every adjustment traceable to named amplifiers (blast radius, propagation speed, structural nature, default exposure, exploitation pattern, institutional criticality). We validate on 20 open-source projects spanning all OWASP Top 10:2025 categories and language risk classes, achieving confidence scores of 0.79-0.97 - precision-recall-derived certainty estimates from bootstrap validation against confirmed CVE incidents in CISA KEV and NVD, not softmax probabilities - and show explainability is guaranteed by architectural design. This addresses the assumed link between network depth and representational power, showing a shallow network with deep reasoning can outperform opaque models in high-stakes cybersecurity where interpretability is non-negotiable.

## Towards GPROF V08: A Machine-Learning Framework for Simulating Passive Microwave Observations of Precipitation

DevFeed: [Towards GPROF V08: A Machine-Learning Framework for Simulating Passive Microwave Observations of Precipitation](<https://devfeed.tech/articles/towards-gprof-v08-a-machine-learning-framework-for-simulating-passive-microwave-observations-of-precipitation-138967.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.19966>)

Author: Simon Pfreundschuh, Christian D. Kummerow, Paula J. Brown

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Neural Network](<https://devfeed.tech/topics/neural-network.md>), [Deep learning](<https://devfeed.tech/topics/deep-learning.md>)

Tags: [3d](<https://devfeed.tech/tags/3d.md>), [algorithm](<https://devfeed.tech/tags/algorithm.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [cnns](<https://devfeed.tech/tags/cnns.md>), [convolutional-neural-networks](<https://devfeed.tech/tags/convolutional-neural-networks.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [data](<https://devfeed.tech/tags/data.md>), [errors](<https://devfeed.tech/tags/errors.md>), [framework](<https://devfeed.tech/tags/framework.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [microwave](<https://devfeed.tech/tags/microwave.md>), [mission](<https://devfeed.tech/tags/mission.md>), [network](<https://devfeed.tech/tags/network.md>), [neural-network](<https://devfeed.tech/tags/neural-network.md>), [neural-networks](<https://devfeed.tech/tags/neural-networks.md>), [next](<https://devfeed.tech/tags/next.md>), [operational](<https://devfeed.tech/tags/operational.md>), [physics-ao-ph](<https://devfeed.tech/tags/physics-ao-ph.md>), [sensors](<https://devfeed.tech/tags/sensors.md>), [simulation](<https://devfeed.tech/tags/simulation.md>), [simulations](<https://devfeed.tech/tags/simulations.md>), [spatial](<https://devfeed.tech/tags/spatial.md>)

### AI overview

Conventional radiative transfer simulations lack the spatial coverage needed to train CNN-based precipitation retrievals across the GPM constellation. The proposed framework uses Satformer to translate passive microwave observations between GPM sensors, generating synthetic observations reported to have lower errors than conventional simulations and to improve GPROF-NN retrieval accuracy. The authors also demonstrate retrievals for the Microwave Imager aboard the Weather System Follow-on Microwave satellite; against independent reference data, these achieve lower instantaneous errors and higher correlations than conventional retrievals. Residual simulation errors can still cause systematic retrieval biases. The framework is presented as a component for extending GPROF-NN to the full constellation in the upcoming GPROF V08 algorithm.

### Source excerpt

arXiv:2611.19966v1 Announce Type: cross Abstract: Spatially resolved precipitation retrievals based on convolutional neural networks (CNNs) can improve precipitation estimates from passive microwave (PMW) observations and will be used to implement the next operational PMW retrieval algorithm for the Global Precipitation Measurement (GPM) mission. However, the conventional radiative transfer simulations used to extend the retrieval algorithm to the various sensors of the GPM constellation do not provide the spatial coverage required to train CNN-based retrievals. To address this, we introduce a machine-learning-based simulation framework that enables the training of the CNN-based GPROF-NN 3D retrieval algorithm for the full GPM constellation. At the core of this framework is Satformer, a new neural-network architecture that can translate PMW observations between any pair of GPM sensors. The resulting synthetic observations exhibit lower errors than conventional radiative transfer simulations and lead to more accurate precipitation retrievals when used to train GPROF-NN precipitation retrievals. We further demonstrate the applicability of the framework to new instruments by developing GPROF-NN retrievals for the Microwave Imager aboard the Weather System Follow-on Microwave satellite. While GPROF-NN retrievals based on the new simulation framework achieve lower instantaneous retrieval errors and higher correlations compared to independent reference data than conventional retrievals, residual simulation errors can introduce systematic biases in the resulting retrievals. The proposed simulation framework provides a key component for extending GPROF-NN to the full GPM constellation as part of the upcoming operational GPROF V08 retrieval algorithm. The results demonstrate the potential of machine-learning-based simulations to support spatially resolved precipitation retrievals across heterogeneous satellite constellations.

## Generalization Bounds for Neural Networks with Sparse Connectivity

DevFeed: [Generalization Bounds for Neural Networks with Sparse Connectivity](<https://devfeed.tech/articles/generalization-bounds-for-neural-networks-with-sparse-connectivity-138867.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.19382>)

Author: Nong Minh Hieu, Antoine Ledent

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Deep neural networks](<https://devfeed.tech/topics/deep-neural-networks.md>), [Deep learning](<https://devfeed.tech/topics/deep-learning.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>)

Tags: [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [neural-networks](<https://devfeed.tech/tags/neural-networks.md>), [sparsity](<https://devfeed.tech/tags/sparsity.md>), [stat-ml](<https://devfeed.tech/tags/stat-ml.md>), [theory](<https://devfeed.tech/tags/theory.md>)

### AI overview

The paper derives non-asymptotic generalization-gap bounds for deep neural networks whose layer-wise weight matrices satisfy ℓp quasi-norm constraints for 0 < p ≤ 1. The result interpolates between norm-based guarantees and parameter-counting bounds; as p approaches zero, it yields square-root dependence on the number of active (nonzero) parameters, which the authors report is tighter than existing parameter-counting approaches. Experiments show milder dependence on the initial architecture in the over-parameterized regime and suggest the bounds capture generalization benefits associated with sparsity learned from natural data.

### Source excerpt

arXiv:2611.19382v1 Announce Type: cross Abstract: Empirical evidence suggests that large neural networks rarely make effective use of all available parameters, with learned solutions exhibiting pronounced sparsity. Despite this ubiquity, existing generalization theory only partially explains how such sparsity influences statistical performance. In this paper, we derive non-asymptotic generalization gap bounds for deep neural networks whose layer-wise weight matrices are restricted by $\ell_p$ quasi-norm ($0

## Deep Learning for Physicists: From Linear Regression to Reasoning Models

DevFeed: [Deep Learning for Physicists: From Linear Regression to Reasoning Models](<https://devfeed.tech/articles/deep-learning-for-physicists-from-linear-regression-to-reasoning-models-139151.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.21295>)

Author: Masanori Hanada

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Deep learning](<https://devfeed.tech/topics/deep-learning.md>), [Neural Network](<https://devfeed.tech/topics/neural-network.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [rlhf](<https://devfeed.tech/topics/rlhf.md>)

Tags: [adversarial](<https://devfeed.tech/tags/adversarial.md>), [attention](<https://devfeed.tech/tags/attention.md>), [backpropagation](<https://devfeed.tech/tags/backpropagation.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [deep-learning](<https://devfeed.tech/tags/deep-learning.md>), [embeddings](<https://devfeed.tech/tags/embeddings.md>), [gpt](<https://devfeed.tech/tags/gpt.md>), [gradient-descent](<https://devfeed.tech/tags/gradient-descent.md>), [hep-lat](<https://devfeed.tech/tags/hep-lat.md>), [hep-th](<https://devfeed.tech/tags/hep-th.md>), [human-feedback](<https://devfeed.tech/tags/human-feedback.md>), [kernel](<https://devfeed.tech/tags/kernel.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [large-language-models](<https://devfeed.tech/tags/large-language-models.md>), [layer](<https://devfeed.tech/tags/layer.md>), [linear-regression](<https://devfeed.tech/tags/linear-regression.md>), [machine](<https://devfeed.tech/tags/machine.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [natural-language](<https://devfeed.tech/tags/natural-language.md>), [natural-language-processing](<https://devfeed.tech/tags/natural-language-processing.md>), [networks](<https://devfeed.tech/tags/networks.md>), [neural](<https://devfeed.tech/tags/neural.md>), [physics-comp-ph](<https://devfeed.tech/tags/physics-comp-ph.md>), [reasoning](<https://devfeed.tech/tags/reasoning.md>)

### AI overview

These lecture notes offer an introductory path through machine learning that uses physical intuition and assumes little prior knowledge. They begin with maximum likelihood, least squares, linear regression, and gradient descent, then cover neural networks, training challenges, regularization, adversarial examples, and kernel methods. The second half moves through NLP--from embeddings, recurrent networks, and attention to Transformers and large language models--covering topics such as instruction tuning, human feedback, hallucination, chain-of-thought reasoning, and test-time scaling. The notes are aimed at UK third- and fourth-year students, claim no originality, and do not cover developments after spring 2025.

### Source excerpt

arXiv:2611.21295v1 Announce Type: new Abstract: These lecture notes are based on an introductory course on neural networks and deep learning that the author taught in the School of Mathematical Sciences at Queen Mary University of London in 2023/24 and 2024/25. The author, however, is a physicist, and it shows: wherever possible, ideas are explained with physical intuition rather than heavy formalism. Readers with a background in physics or mathematics who want a self-contained, introductory route into modern machine learning may therefore find these notes particularly friendly. The notes start from maximum likelihood, least squares, linear regression and (stochastic) gradient descent. They then cover neural networks, backpropagation, the vanishing gradient problem, batch and layer normalization, overfitting and regularization, adversarial examples, and the kernel method. The second half turns to natural language processing: word embeddings, recurrent networks and LSTMs, the attention mechanism and Transformers, and large language models, including scaling laws, the history of GPT, instruction tuning, reinforcement learning from human feedback, hallucination, alignment faking, chain-of-thought reasoning and test-time scaling. The emphasis on large language models reflects requests from the students. The notes are aimed at third- and fourth-year students in the UK system. Because the course was also taken by students from other departments, very little is assumed: even basic material such as matrix multiplication is explained from scratch, so readers familiar with it can safely skip such parts. No originality is claimed; the aim is a coherent, intuitive path through the material. Developments after spring 2025 are not covered, but the basic ideas emphasized here are unlikely to go out of date soon.

[Next page](<https://devfeed.tech/topics/deep-learning.md?cursor=WyIyMDI2LTExLTE3VDA1OjAwOjAwKzAwOjAwIiwgIjZhNmRhZDI0LWQ5YzMtNGM3Zi04MjYyLWFjNDI1ODFmODY1MCJd>)