# mlp

MLP (multilayer perceptron) is a feedforward neural network composed of connected dense layers and used for classification, regression, and function approximation.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## Hierarchical Personalized Federated Learning for Fine-Grained Intrusion Detection: A Multi-Class Analysis in Heterogeneous Attack Environments

DevFeed: [Hierarchical Personalized Federated Learning for Fine-Grained Intrusion Detection: A Multi-Class Analysis in Heterogeneous Attack Environments](<https://devfeed.tech/articles/hierarchical-personalized-federated-learning-for-fine-grained-intrusion-detection-a-multi-class-analysis-in-heterogeneous-attack-environments-141478.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.21970>)

Author: Abul Kalam Azad, Travis Atkison

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Cryptography and Security](<https://devfeed.tech/sources/arxiv-cryptography-and-security.md>)

Topics: [Federated Learning](<https://devfeed.tech/topics/federated-learning.md>), [NVIDIA Flare](<https://devfeed.tech/topics/nvidia-flare.md>), [threat detection](<https://devfeed.tech/topics/threat-detection.md>), [mlp](<https://devfeed.tech/topics/mlp.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Security](<https://devfeed.tech/topics/security.md>)

Tags: [aggregation](<https://devfeed.tech/tags/aggregation.md>), [attack](<https://devfeed.tech/tags/attack.md>), [classification](<https://devfeed.tech/tags/classification.md>), [cs-cr](<https://devfeed.tech/tags/cs-cr.md>), [dataset](<https://devfeed.tech/tags/dataset.md>), [evaluations](<https://devfeed.tech/tags/evaluations.md>), [federated-learning](<https://devfeed.tech/tags/federated-learning.md>), [intrusion-detection](<https://devfeed.tech/tags/intrusion-detection.md>), [mlp](<https://devfeed.tech/tags/mlp.md>), [nvidia-flare](<https://devfeed.tech/tags/nvidia-flare.md>), [out-of-distribution-generalization](<https://devfeed.tech/tags/out-of-distribution-generalization.md>), [overhead](<https://devfeed.tech/tags/overhead.md>), [personalization](<https://devfeed.tech/tags/personalization.md>), [privacy](<https://devfeed.tech/tags/privacy.md>), [security](<https://devfeed.tech/tags/security.md>)

### AI overview

The abstract reports that hierarchical personalized federated learning (HPFL), combining multi-tier aggregation with client-level personalization, improved fine-grained intrusion detection on CICIDS2017 using MLP models in NVIDIA FLARE. In the 8-class local evaluation, HPFL reached 0.8324 macro F1, above FedAvg (0.7176) and FedProx (0.5624); its micro F1 was 0.9901. Cross-client 4-class micro F1 was 0.9294, compared with 0.9140 for standalone local models. The study also estimates 194.4 MB transmission overhead for HPFL versus 380.3 MB for conventional FL in the 8-class task. These abstract-level results suggest benefits under heterogeneous attack distributions; they do not establish performance beyond the reported dataset and evaluations.

### Source excerpt

arXiv:2611.21970v1 Announce Type: new Abstract: Intrusion Detection Systems (IDS) play a vital role in protecting modern network infrastructures. Although Federated Learning (FL) enables collaborative training without sharing raw network traffic, global models struggle with non-IID attack distributions. Moreover, practical security operations require fine-grained attack classification for targeted threat responses. To address these challenges, this paper investigates the convergence dynamics and performance of Hierarchical Personalized Federated Learning (HPFL), which combines multi-tier aggregation with client-level personalization, for 8-class, 6-class, and 4-class intrusion detection using Multi-Layer Perceptron (MLP) models implemented in NVIDIA FLARE on the CICIDS2017 dataset. Experimental results show that personalized HPFL models achieve micro F1-scores up to 0.9911 in the 6-class task, while the overall local accuracy reaches 0.9975 in the 8-class task. On 8-class local test sets, HPFL achieves a micro F1-score of 0.9901 and a macro F1-score of 0.8324, outperforming FedAvg (0.7176) and FedProx (0.5624) in macro F1. Cross-client evaluations demonstrate out-of-distribution generalization, with HPFL achieving a 4-class micro F1-score of 0.9294 compared with 0.9140 for standalone local models. Furthermore, distance-weighted network analysis estimates a 48.9% reduction in transmission overhead compared with conventional FL (194.4 MB vs. 380.3 MB for the 8-class task). These findings demonstrate HPFLs potential as an accurate, privacy-preserving, and communication-efficient approach to fine-grained intrusion detection in heterogeneous networks.

## OCCAM: Improved Single-Cell Representation Learning and Perturbation Effect Prediction

DevFeed: [OCCAM: Improved Single-Cell Representation Learning and Perturbation Effect Prediction](<https://devfeed.tech/articles/occam-improved-single-cell-representation-learning-and-perturbation-effect-prediction-139939.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.21959>)

Author: Ihab Bendidi, Kian Kenyon-Dean, Alisandra K. Denton, Auguste Genovesio, Emmanuel Noutahi

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Representation learning](<https://devfeed.tech/topics/representation-learning.md>), [mlp](<https://devfeed.tech/topics/mlp.md>), [few-shot](<https://devfeed.tech/topics/few-shot.md>), [Neural Network](<https://devfeed.tech/topics/neural-network.md>)

Tags: [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [denoising](<https://devfeed.tech/tags/denoising.md>), [few-shot](<https://devfeed.tech/tags/few-shot.md>), [interaction](<https://devfeed.tech/tags/interaction.md>), [mlp](<https://devfeed.tech/tags/mlp.md>), [prediction](<https://devfeed.tech/tags/prediction.md>), [q-bio-qm](<https://devfeed.tech/tags/q-bio-qm.md>), [representation-learning](<https://devfeed.tech/tags/representation-learning.md>), [single-cell](<https://devfeed.tech/tags/single-cell.md>)

### AI overview

Occam is an MLP framework that combines denoising-learned cell representations with gene interaction, text, and protein-sequence features to predict genetic perturbation effects. The abstract reports the highest aggregate representation scores among evaluated methods for perturbation information and cell-type preservation with batch integration. In few-shot tests where perturbation and cell context were separately seen but not together, it captured measured response patterns better than four retrained competitors and direct transfer from other cell lines. Without fine-tuning, it also best distinguished perturbations in held-out cell lines across seen and unseen assays, though accurate expression-level prediction remained challenging. For double perturbations, composing learned single-gene responses reduced expression error versus competing models and simply summing single-gene effects, including for strongly and weakly additive responses.

### Source excerpt

arXiv:2611.21959v1 Announce Type: cross Abstract: Predicting the effects of genetic perturbations requires models that generalize beyond observed cell contexts and gene combinations. We introduce Occam, a multilayer perceptron (MLP) framework for representation learning and perturbation-effect prediction. Occam combines cell representations learned through denoising with gene interaction, text, and protein-sequence features. It achieves the highest aggregate representation scores for perturbation information and cell-type preservation with batch integration. In cross-context few-shot prediction, a perturbation and a cell context are each observed during training, but never together. Occam better captures the measured response patterns for these unobserved combinations than four retrained competitors or direct transfer of the same perturbation's measured effects from other training cell lines. Without fine-tuning, Occam best distinguishes perturbations among the evaluated models and simple baselines in cell lines held out from predictor training, across seen and unseen assays. Predicting expression levels accurately in these settings remains challenging. For double perturbations, which target two genes simultaneously, Occam composes learned single-gene responses to better capture the combined response and reduce expression error compared with competing models and summed single-gene effects. These gains extend to both strongly additive and weakly additive responses. Together, these results support simple MLPs for cell representation learning and perturbation-effect prediction across contexts and gene combinations.

## Gradient Boosting vs. Random Forest vs. Neural Networks: A 2026 Performance Benchmark

DevFeed: [Gradient Boosting vs. Random Forest vs. Neural Networks: A 2026 Performance Benchmark](<https://devfeed.tech/articles/gradient-boosting-vs-random-forest-vs-neural-networks-a-2026-performance-benchmark-137767.md>)

Original publisher: [Read original article](<https://machinelearningmastery.com/gradient-boosting-vs-random-forest-vs-neural-networks-a-2026-performance-benchmark/>)

Author: Vinod Chugani

Published: 2026-11-17T11:00:00Z

Content type: comparison

Language: en

Sources: [MachineLearningMastery.com](<https://devfeed.tech/sources/machinelearningmastery-com.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Benchmark](<https://devfeed.tech/topics/benchmark.md>), [Neural Network](<https://devfeed.tech/topics/neural-network.md>), [AI Foundation Models](<https://devfeed.tech/topics/ai-foundation-models.md>), [Deep learning](<https://devfeed.tech/topics/deep-learning.md>), [Hyperparameter optimization](<https://devfeed.tech/topics/hyperparameter-optimization.md>), [Transformer](<https://devfeed.tech/topics/transformer.md>), [catboost](<https://devfeed.tech/topics/catboost.md>), [mlp](<https://devfeed.tech/topics/mlp.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>)

Tags: [2026](<https://devfeed.tech/tags/2026.md>), [algorithm](<https://devfeed.tech/tags/algorithm.md>), [algorithms](<https://devfeed.tech/tags/algorithms.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [audio](<https://devfeed.tech/tags/audio.md>), [backpropagation](<https://devfeed.tech/tags/backpropagation.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [catboost](<https://devfeed.tech/tags/catboost.md>), [classification](<https://devfeed.tech/tags/classification.md>), [cnns](<https://devfeed.tech/tags/cnns.md>), [compare](<https://devfeed.tech/tags/compare.md>), [compute](<https://devfeed.tech/tags/compute.md>), [context-window](<https://devfeed.tech/tags/context-window.md>), [cpu](<https://devfeed.tech/tags/cpu.md>), [foundation-models](<https://devfeed.tech/tags/foundation-models.md>), [gradient-boosting](<https://devfeed.tech/tags/gradient-boosting.md>), [neural-networks](<https://devfeed.tech/tags/neural-networks.md>), [performance](<https://devfeed.tech/tags/performance.md>), [practical-machine-learning](<https://devfeed.tech/tags/practical-machine-learning.md>), [requirements](<https://devfeed.tech/tags/requirements.md>), [speed](<https://devfeed.tech/tags/speed.md>)

### AI overview

The excerpt describes two regimes for tabular data: tabular foundation models (TFMs) lead TabArena accuracy results on small-to-medium datasets, while gradient boosting remains stronger for large-scale data and regression. TabPFN-3.5 is reported to support up to 1 million rows and 20,000 features, but TFM limits vary and inference requires GPU memory to hold data in context. XGBoost, LightGBM, and CatBoost offer CPU-based training and low-latency inference, making them practical under compute or latency constraints. Random forest is a forgiving, quickly tuned baseline, especially with noisy data or limited tuning resources. Specialized networks have narrower advantages: TabR can outperform XGBoost on some classification tasks, while TabM and RealMLP reach parity with boosted trees on large datasets when carefully tuned and ensembled. For images, text, and audio, transformer architectures lead.

### Source excerpt

In this article, you will learn how gradient boosting, random forest, neural networks, and tabular foundation models compare on accuracy, speed, and data requirements in...

## Improving X-Ray Baggage Instance Segmentation with Shared Point-Based Mask Refinement

DevFeed: [Improving X-Ray Baggage Instance Segmentation with Shared Point-Based Mask Refinement](<https://devfeed.tech/articles/improving-x-ray-baggage-instance-segmentation-with-shared-point-based-mask-refinement-137578.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.20476>)

Author: Abdulaziz Alqahtani, Qasim Umer

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Image](<https://devfeed.tech/topics/image.md>), [mlp](<https://devfeed.tech/topics/mlp.md>), [Transformer](<https://devfeed.tech/topics/transformer.md>)

Tags: [attention](<https://devfeed.tech/tags/attention.md>), [boundary](<https://devfeed.tech/tags/boundary.md>), [cascade](<https://devfeed.tech/tags/cascade.md>), [cnn](<https://devfeed.tech/tags/cnn.md>), [computer-vision](<https://devfeed.tech/tags/computer-vision.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [detection](<https://devfeed.tech/tags/detection.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [independent](<https://devfeed.tech/tags/independent.md>), [mask](<https://devfeed.tech/tags/mask.md>), [mlp](<https://devfeed.tech/tags/mlp.md>), [point](<https://devfeed.tech/tags/point.md>), [precision](<https://devfeed.tech/tags/precision.md>), [quality](<https://devfeed.tech/tags/quality.md>), [ray](<https://devfeed.tech/tags/ray.md>), [supervision](<https://devfeed.tech/tags/supervision.md>), [target](<https://devfeed.tech/tags/target.md>), [training](<https://devfeed.tech/tags/training.md>), [transformer](<https://devfeed.tech/tags/transformer.md>)

### AI overview

Shared point-based refinement improved mask segmentation in the tested Cascade Mask R-CNN on the official PIXray split. The system uses a ResNet-101 backbone and attention-based feature pyramid, progressively refining uncertain mask points from 28x28 to 112x112 with one module shared across cascade stages and refinement levels. Across three seeds, the transformer and MLP variants reached mean mask AP of 71.79 and 72.47, versus 70.50 without refinement; the MLP scored highest among the evaluated configurations. Both improved boundary agreement on commonly detected objects, but detection diagnostics did not establish a corresponding positive gain. The study used existing target-instance masks without additional occluder supervision; independent X-ray datasets are needed to assess generalization.

### Source excerpt

arXiv:2611.20476v1 Announce Type: cross Abstract: Overlapping projections and complex object contours make prohibited items difficult to delineate in X-ray baggage images. This study investigates shared point-based mask refinement to improve instance segmentation within Cascade Mask R-CNN. The framework combines a ResNet-101 backbone with an attention-based feature pyramid and progressively updates uncertain mask locations, increasing mask resolution from $28 \times 28$ to $112 \times 112$. A single refinement module is shared across cascade stages and refinement levels, with transformer and multilayer perceptron (MLP) variants used to examine the role of interactions among selected points. Ten configurations were evaluated on the official PIXray split across three training seeds using the original target-instance annotations for evaluation. The transformer and MLP refiners achieved mean mask average precision (AP) of 71.79 and 72.47, respectively, compared with 70.50 for the unrefined detector, yielding gains of 1.29 and 1.97 AP points. The MLP achieved the highest mean mask AP among the evaluated configurations. Both variants also improved boundary agreement on commonly detected objects, supporting improved mask delineation, while detection diagnostics did not establish a corresponding positive gain. These findings demonstrate that shared point-based refinement can improve segmentation quality in the tested cascade architecture using existing target-mask annotations, without additional occluder supervision. Evaluation on independent X-ray datasets is needed to determine how well these improvements generalize.

## Warm Starting Neural Dynamics Models From Few Jacobian Samples

DevFeed: [Warm Starting Neural Dynamics Models From Few Jacobian Samples](<https://devfeed.tech/articles/warm-starting-neural-dynamics-models-from-few-jacobian-samples-137701.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.21465>)

Author: Maxime Rochkoulets, Joppe De Jonghe, Mariya Ishteva

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>), [Neural Network](<https://devfeed.tech/topics/neural-network.md>), [mlp](<https://devfeed.tech/topics/mlp.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>)

Tags: [algorithm](<https://devfeed.tech/tags/algorithm.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-sy](<https://devfeed.tech/tags/cs-sy.md>), [eess-sy](<https://devfeed.tech/tags/eess-sy.md>), [efficiency](<https://devfeed.tech/tags/efficiency.md>), [neural-network](<https://devfeed.tech/tags/neural-network.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>)

### AI overview

TDec-Init warm-starts neural dynamics models for model-based reinforcement learning from transition Jacobians. It decomposes a tensor of stacked Jacobian samples to recover network weights and flexible nonlinearities, then uses them to initialize a feedforward network. In low-data benchmark settings, the authors report better performance than random initialization and value training, and performance matching or exceeding Sobolev training, with fewer training steps between episodes. The supplied text contains only the abstract, so it does not provide benchmark details or broader evaluation results.

### Source excerpt

arXiv:2611.21465v1 Announce Type: cross Abstract: Model-based reinforcement learning (MBRL) leverages a model of the environment, used for planning and generating samples without interacting with the real world. Motivated by evidence that informed initialization significantly improves the efficiency of MBRL, we propose TDec-Init, an algorithm to warm-start neural dynamics models using transition Jacobians. Using a decoupling algorithm based on the decomposition of a tensor of stacked Jacobian samples, we recover weights and flexible nonlinearities, and use them to initialize a feedforward neural network. In the low-data regime, we show on several benchmark environments that our approach outperforms random initialization and value training, and either matches or outperforms Sobolev training, while requiring fewer training steps between episodes.

## Axiomatic Positivstellensaetze for Function Algebras over Ordered Fields

DevFeed: [Axiomatic Positivstellensaetze for Function Algebras over Ordered Fields](<https://devfeed.tech/articles/axiomatic-positivstellensaetze-for-function-algebras-over-ordered-fields-137280.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2609.08736>)

Author: Nayoon Kim, Allen Gehret, Shenyuan Ma, Jakub Marecek

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [polynomials](<https://devfeed.tech/topics/polynomials.md>), [mlp](<https://devfeed.tech/topics/mlp.md>)

Tags: [activation](<https://devfeed.tech/tags/activation.md>), [algebra](<https://devfeed.tech/tags/algebra.md>), [certificate](<https://devfeed.tech/tags/certificate.md>), [certificates](<https://devfeed.tech/tags/certificates.md>), [closure](<https://devfeed.tech/tags/closure.md>), [continuous](<https://devfeed.tech/tags/continuous.md>), [core](<https://devfeed.tech/tags/core.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cubic](<https://devfeed.tech/tags/cubic.md>), [field](<https://devfeed.tech/tags/field.md>), [fields](<https://devfeed.tech/tags/fields.md>), [fixed](<https://devfeed.tech/tags/fixed.md>), [framework](<https://devfeed.tech/tags/framework.md>), [function](<https://devfeed.tech/tags/function.md>), [functions](<https://devfeed.tech/tags/functions.md>), [global](<https://devfeed.tech/tags/global.md>), [lower-bounds](<https://devfeed.tech/tags/lower-bounds.md>), [networks](<https://devfeed.tech/tags/networks.md>), [neural](<https://devfeed.tech/tags/neural.md>), [neural-networks](<https://devfeed.tech/tags/neural-networks.md>), [number](<https://devfeed.tech/tags/number.md>)

### AI overview

The paper gives an axiomatic framework for constructive strict and weak Positivstellensätze over ordered fields and algebras of functions, extending positivity certificates beyond polynomials and smooth functions. With five auxiliary function symbols and closure axioms, it constructs witnesses by substituting the problem data into fixed terms depending only on the number of constraints. The framework recovers Fischer's results for definable C^r functions and covers additional algebras, including those with arbitrary auxiliary functions, continuous functions with criteria tied to common activation functions, and continuous functions valued in fields such as Q that are not closed under square roots. It derives certificates for lower bounds and global optimality; expanded term length is cubic in the number of constraints for strict certificates and quadratic for weak ones.

### Source excerpt

arXiv:2609.08736v2 Announce Type: replace Abstract: Positivstellensaetze furnish algebraic certificates of positivity, and hence of lower bounds and of global optimality, but the classical ones concern polynomials, whereas many problems of current interest, among them the training of neural networks, involve functions that are neither polynomial nor smooth. We isolate an axiomatic core of the constructive Positivstellensaetze of Fischer. The setting consists of an ordered field expanded by five auxiliary function symbols, subject to a few first-order axioms, together with an algebra of functions on an arbitrary set, subject to closure axioms. In this setting we prove a strict and a weak Positivstellensatz in which the witnesses are obtained by substituting the data into fixed terms that depend only on the number of constraints. The framework recovers Fischer's theorems for definable $C^r$-functions, and yields new instances: algebras of definable functions with arbitrary auxiliary functions, such as the binary step function; algebras of continuous functions, for which we give simple sufficient criteria and auxiliary functions derived from common activation functions; and algebras of continuous functions with values in an ordered field that is not closed under square roots, such as $\mathbb{Q}$. We derive certificates of lower bounds and of global optimality, and show that their expanded term length is cubic in the number of constraints for the strict certificate and quadratic for the weak one.

## Are Kolmogorov-Arnold Networks (KANs) Suitable for Imbalanced Data Classification?

DevFeed: [Are Kolmogorov-Arnold Networks (KANs) Suitable for Imbalanced Data Classification?](<https://devfeed.tech/articles/are-kolmogorov-arnold-networks-kans-suitable-for-imbalanced-data-classification-138720.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2507.14121>)

Author: Pankaj Yadav, Vivek Vijay

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Advanced AI workflows](<https://devfeed.tech/topics/advanced-ai-workflows.md>), [mlp](<https://devfeed.tech/topics/mlp.md>), [Feature Engineering](<https://devfeed.tech/topics/feature-engineering.md>)

Tags: [bias](<https://devfeed.tech/tags/bias.md>), [classification](<https://devfeed.tech/tags/classification.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [distribution](<https://devfeed.tech/tags/distribution.md>), [effective](<https://devfeed.tech/tags/effective.md>), [imbalanced-data](<https://devfeed.tech/tags/imbalanced-data.md>), [interpretability](<https://devfeed.tech/tags/interpretability.md>), [kan](<https://devfeed.tech/tags/kan.md>), [latency](<https://devfeed.tech/tags/latency.md>), [mlp](<https://devfeed.tech/tags/mlp.md>), [rank](<https://devfeed.tech/tags/rank.md>), [recover](<https://devfeed.tech/tags/recover.md>), [sparsity](<https://devfeed.tech/tags/sparsity.md>), [structured](<https://devfeed.tech/tags/structured.md>), [training](<https://devfeed.tech/tags/training.md>), [xgboost](<https://devfeed.tech/tags/xgboost.md>)

### AI overview

Across ten imbalanced tabular datasets, KANs showed stronger majority-class bias than capacity-matched MLPs under native class distributions, while Focal Loss brought negligible improvement. SMOTE-Tomek resampling restored structured minority-class spline activations and gave KANs the best cohort-wide rank, with a significant gain over unmitigated KANs but not over resampled baselines. KAN activation profiles provide native insight into these representational changes. That interpretability comes with substantially higher training overhead than a matched MLP across all conditions, and no strategy-dependent latency reduction; the results therefore support KANs conditionally, especially with data-level resampling.

### Source excerpt

arXiv:2507.14121v2 Announce Type: replace Abstract: Kolmogorov-Arnold Networks (KANs) parameterize each connection as a learnable univariate B-spline, offering native feature-level interpretability. Whether this locality helps or hinders under class imbalance, where minority sparsity may leave spline edges without enough local observations to update reliably, remains unexamined. We evaluate KANs against capacity-matched Multilayer Perceptrons (MLPs) and XGBoost across ten imbalanced tabular datasets under three conditions: native class distribution, Focal Loss, and SMOTE-Tomek resampling, within a parameter-parity, two-phase framework. Under the native distribution, KAN spline activations collapse in minority-sparse regions, showing stronger majority-class bias than the matched MLP. Focal Loss yields negligible improvement, as it reweights gradients without providing the localized observations needed for discriminative curvature. Conversely, SMOTE-Tomek resampling lets spline activations recover structured minority-class representations, achieving the best cohort-wide rank and a large, statistically significant gain over unmitigated KANs, though not over resampled baselines. KAN's native spline interpretability directly reveals these dynamics via learned activation profiles, proving representational change rather than threshold recalibration without post-hoc attribution. However, these interpretability gains incur substantially higher training overhead than a parameter-matched MLP across all conditions, with no strategy-dependent latency reduction. Thus, KANs are conditionally suitable for imbalanced tabular classification: effective and interpretable with data-level resampling, yet costly relative to simpler alternatives without it.

## Zero-Shot Weight-Space Programming for Prediction and Classification of Target Variables in Structured Datasets

DevFeed: [Zero-Shot Weight-Space Programming for Prediction and Classification of Target Variables in Structured Datasets](<https://devfeed.tech/articles/zero-shot-weight-space-programming-for-prediction-and-classification-of-target-variables-in-structured-datasets-137426.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.19292>)

Author: Eduardo C. Garrido-Merch\\'an

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [mlp](<https://devfeed.tech/topics/mlp.md>), [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [few-shot-learning](<https://devfeed.tech/topics/few-shot-learning.md>)

Tags: [classification](<https://devfeed.tech/tags/classification.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [gradient-descent](<https://devfeed.tech/tags/gradient-descent.md>), [hidden](<https://devfeed.tech/tags/hidden.md>), [inference](<https://devfeed.tech/tags/inference.md>), [large-language-model](<https://devfeed.tech/tags/large-language-model.md>), [linear](<https://devfeed.tech/tags/linear.md>), [network](<https://devfeed.tech/tags/network.md>), [prediction](<https://devfeed.tech/tags/prediction.md>), [problems](<https://devfeed.tech/tags/problems.md>), [regression](<https://devfeed.tech/tags/regression.md>), [rotation](<https://devfeed.tech/tags/rotation.md>), [synthetic](<https://devfeed.tech/tags/synthetic.md>), [weight-space-programming](<https://devfeed.tech/tags/weight-space-programming.md>), [zero-shot](<https://devfeed.tech/tags/zero-shot.md>)

### AI overview

The study tests whether an LLM can infer a supervised dataset's target and encode it directly as every weight and bias in a feed-forward network, without optimization, code execution, or a chance to evaluate the network first. Across toy, rotated synthetic, benchmark, and engineering regression problems, its frozen networks match the closed-form optimum on linear tasks and reach the generating process's irreducible error on three nonlinear tasks; on two of those they outperform fitted baselines, including the same architecture trained by gradient descent. It fails on two tasks where the target structure is oblique to the analyzed coordinates. Additional constructors reveal distinct failures: some produce no parameters on the hardest tasks, while another always responds but can perform worse than predicting the mean. A separate constructor solves one oblique-hinge task, and linear-separator training error predicts when a hidden layer helps better than the tested composite complexity index.

### Source excerpt

arXiv:2611.19292v1 Announce Type: new Abstract: Weight-space programming fixes every parameter of a feed-forward network by hand, with no gradient descent anywhere in the construction. Every demonstration so far starts from a target function handed to the model as a rule system, making it a compilation problem. A supervised dataset withholds that specification, so the procedure must do the statistical inference itself and then express the result in the parameterisation it was given. We ask whether a large language model can do both. It is shown a small dataset in full, with the architecture of a feed-forward network, and writes out every weight and bias as a literal number, with no optimizer, no gradient, no code execution, and no chance to evaluate it first. Each construction is sealed and pre-registers its expected score. The graded suite runs from two toy problems, through nine synthetic problems disguised by a hidden rotation, to the classical benchmarks and one real engineering regression. The frozen network matches the closed-form optimum wherever that optimum is linear, reaches the irreducible error of the generating process on three rungs where it is not, and beats every fitted baseline on two of those, including the same architecture trained by gradient descent. It fails on two rungs, both times because the structure to be found is oblique to the coordinates it analysed. Three further constructors separate two failure modes a single score conflates: one returns no parameter vector at all on the hardest rungs, another delivers everywhere while writing networks worse than predicting the mean. One of those failures proves avoidable: another constructor solves the oblique-hinge rung by a margin that survives multiple-comparison correction. What predicts whether the hidden layer earns its keep is the training error of a linear separator, which the composite complexity index misses.

## Representational Geometry as a Diagnostic for Adversarial Robustness

DevFeed: [Representational Geometry as a Diagnostic for Adversarial Robustness](<https://devfeed.tech/articles/representational-geometry-as-a-diagnostic-for-adversarial-robustness-135819.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.17684>)

Author: Brandon Ismalej, Alex Cloninger, Gal Mishne

Published: 2026-11-16T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Cryptography and Security](<https://devfeed.tech/sources/arxiv-cryptography-and-security.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Neural Network](<https://devfeed.tech/topics/neural-network.md>), [mlp](<https://devfeed.tech/topics/mlp.md>), [Adversarial attacks](<https://devfeed.tech/topics/adversarial-attacks.md>)

Tags: [adversarial](<https://devfeed.tech/tags/adversarial.md>), [attack](<https://devfeed.tech/tags/attack.md>), [auditing](<https://devfeed.tech/tags/auditing.md>), [cs-cr](<https://devfeed.tech/tags/cs-cr.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [diagnostic](<https://devfeed.tech/tags/diagnostic.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [mnist](<https://devfeed.tech/tags/mnist.md>), [nearest-neighbor](<https://devfeed.tech/tags/nearest-neighbor.md>), [network](<https://devfeed.tech/tags/network.md>), [neural-networks](<https://devfeed.tech/tags/neural-networks.md>), [points](<https://devfeed.tech/tags/points.md>), [robust](<https://devfeed.tech/tags/robust.md>), [robustness](<https://devfeed.tech/tags/robustness.md>), [training](<https://devfeed.tech/tags/training.md>)

### AI overview

The study finds that adversarial training changes how MNIST-trained multilayer perceptrons organize internal representations, and that representational stability can rank defenses differently from robust accuracy. It compares three adversarial training regimes with a vanilla network, using M-PHATE to visualize data geometry across training epochs. For penultimate-layer representations, it measures clean-to-perturbed displacement, maximum mean discrepancy, and nearest-neighbor recovery. The abstract presents these representation geometries and training trajectories as additional diagnostics for auditing robustness; it does not establish that they replace accuracy-under-attack measurements.

### Source excerpt

arXiv:2611.17684v1 Announce Type: new Abstract: Adversarial training is a common technique for making neural networks more robust to small, intentionally crafted input perturbations, and robustness is commonly measured through accuracy under attack. By exploring inter-network representations across different adversarial training objectives, we can analyze whether multiple networks organize their latent representations similarly. We study Multilayer Perceptrons trained on MNIST under three adversarial training regimes and compare the resulting population training dynamics and geometry to those of a vanilla network. We use M-PHATE to embed data samples across training epochs and examine how the geometry of the data changes during training under adversarial objectives. On penultimate-layer representations we measure displacement between clean and perturbed points, distributional distance through maximum mean discrepancy, and nearest-neighbor recovery. Our visualizations and quantitative analyses show that adversarial training leads to structural changes in how networks organize internal data representations, and robust accuracy and representational stability can rank the same defenses in opposite ways. These findings support a discussion of re-evaluating how practitioners approach the auditing of model robustness by treating latent representations and their training trajectories as diagnostic artifacts across sets of networks.

## Asymptotic Margins and Feature Confinement in Conic Classification Heads under Softmax Cross-Entropy

DevFeed: [Asymptotic Margins and Feature Confinement in Conic Classification Heads under Softmax Cross-Entropy](<https://devfeed.tech/articles/asymptotic-margins-and-feature-confinement-in-conic-classification-heads-under-softmax-cross-entropy-135911.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.18507>)

Author: Omer Nezih Gerek

Published: 2026-11-16T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [scikit-learn](<https://devfeed.tech/topics/scikit-learn.md>), [mlp](<https://devfeed.tech/topics/mlp.md>)

Tags: [bounded](<https://devfeed.tech/tags/bounded.md>), [cifar-10](<https://devfeed.tech/tags/cifar-10.md>), [classification](<https://devfeed.tech/tags/classification.md>), [cross-entropy](<https://devfeed.tech/tags/cross-entropy.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [eess-sp](<https://devfeed.tech/tags/eess-sp.md>), [feature](<https://devfeed.tech/tags/feature.md>), [margins](<https://devfeed.tech/tags/margins.md>), [softmax](<https://devfeed.tech/tags/softmax.md>)

### AI overview

Bounded class regions do not necessarily keep learned features bounded under softmax cross-entropy. The paper defines an asymptotic margin, computable by a second-order cone program, to compare each class score's growth with competing scores. Positive margin means the class loss approaches zero only as features diverge; negative margin makes the loss coercive, with minimizers in an explicit ball. For scores combining a linear term and a Euclidean norm, class regions are bounded by a quadric of revolution whose eccentricity is the reciprocal of a learned ratio. The analysis extends to other norms, including polyhedral conic functions. In reported CIFAR-10 and CIFAR-100 experiments, all 1,210 cells across 67 trained conic heads had positive margins, while a prototype head kept nine of ten cells bounded.

### Source excerpt

arXiv:2611.18507v1 Announce Type: new Abstract: Classification heads built on conic functions give each class a bounded region of the feature space, and such regions are used to reject samples of unseen classes. We ask whether these regions also keep the features bounded when the head is trained with softmax cross-entropy. For a head whose class scores combine a linear term with a Euclidean norm, we first show that the region of each class is bounded by a quadric of revolution, whose eccentricity is the reciprocal of one learned ratio. We then attach to each class an asymptotic margin, which compares the growth of its score with the growth of all the other scores and can be computed by a second-order cone program. The loss of a class has infimum zero exactly when its margin is positive, and it then approaches zero only along diverging features. When the margin is negative, the loss is coercive and its minimizers lie in an explicit ball. Since softmax depends only on the differences between scores, a class with a bounded region can still have a positive margin, so bounded regions do not confine the features. The result extends to other norms, including the polyhedral conic functions. On CIFAR-10 and CIFAR-100, all 1210 cells of 67 trained conic heads have positive margins, whereas a prototype head keeps nine of its ten cells bounded, as the theory predicts.

## Benign Overfitting for Regression with Trained Two-Layer ReLU Networks

DevFeed: [Benign Overfitting for Regression with Trained Two-Layer ReLU Networks](<https://devfeed.tech/articles/benign-overfitting-for-regression-with-trained-two-layer-relu-networks-134943.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2410.06191>)

Author: Junhyung Park, Patrick Bloebaum, Shiva Prasad Kasiviswanathan

Published: 2026-11-16T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [mlp](<https://devfeed.tech/topics/mlp.md>), [linear-regression](<https://devfeed.tech/topics/linear-regression.md>)

Tags: [bounded](<https://devfeed.tech/tags/bounded.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [flow](<https://devfeed.tech/tags/flow.md>), [kernel](<https://devfeed.tech/tags/kernel.md>), [overfitting](<https://devfeed.tech/tags/overfitting.md>), [regression](<https://devfeed.tech/tags/regression.md>), [relu](<https://devfeed.tech/tags/relu.md>), [risk](<https://devfeed.tech/tags/risk.md>), [stat-ml](<https://devfeed.tech/tags/stat-ml.md>), [two-layer](<https://devfeed.tech/tags/two-layer.md>)

### AI overview

For least-squares regression trained by gradient flow, the paper reports benign overfitting in finite-width, two-layer fully connected ReLU networks within the neural tangent kernel regime. Its generalization result assumes only that the regression function and noise are bounded. The authors analyze excess risk by separating estimation and approximation errors, treating gradient flow as an implicit regularizer; under the same setting, they also establish that the trained network overfits the data.

### Source excerpt

arXiv:2410.06191v2 Announce Type: replace-cross Abstract: We study the least-square regression problem with a two-layer fully-connected neural network, with ReLU activation function, trained by gradient flow. Our first result is a generalization result, that requires no assumptions on the underlying regression function or the noise other than that they are bounded. We operate in the neural tangent kernel regime, and our generalization result is developed via a decomposition of the excess risk into estimation and approximation errors, viewing gradient flow as an implicit regularizer. This decomposition in the context of neural networks is a novel perspective of gradient descent, and helps us avoid uniform convergence traps. In this work, we also establish that under the same setting, the trained network overfits to the data. Together, these results, establishes the first result on benign overfitting for finite-width ReLU networks for arbitrary regression functions.

## Reinforcement Learning for Inventory Control under Regime-Switching Demand: A Controlled Comparison of LSTM, MLP, and (s,S) Policies

DevFeed: [Reinforcement Learning for Inventory Control under Regime-Switching Demand: A Controlled Comparison of LSTM, MLP, and (s,S) Policies](<https://devfeed.tech/articles/reinforcement-learning-for-inventory-control-under-regime-switching-demand-a-controlled-comparison-of-lstm-mlp-and-s-s-policies-133051.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.15945>)

Author: Alex Haines, Nathanael L. Baisa

Published: 2026-11-13T05:00:00Z

Content type: comparison

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [mlp](<https://devfeed.tech/topics/mlp.md>)

Tags: [comparison](<https://devfeed.tech/tags/comparison.md>), [control](<https://devfeed.tech/tags/control.md>), [credit-assignment](<https://devfeed.tech/tags/credit-assignment.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [custom](<https://devfeed.tech/tags/custom.md>), [daily](<https://devfeed.tech/tags/daily.md>), [deep-reinforcement-learning](<https://devfeed.tech/tags/deep-reinforcement-learning.md>), [demand](<https://devfeed.tech/tags/demand.md>), [emissions](<https://devfeed.tech/tags/emissions.md>), [environment](<https://devfeed.tech/tags/environment.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [gap](<https://devfeed.tech/tags/gap.md>), [hidden](<https://devfeed.tech/tags/hidden.md>), [horizon](<https://devfeed.tech/tags/horizon.md>), [inventory](<https://devfeed.tech/tags/inventory.md>), [lead-time](<https://devfeed.tech/tags/lead-time.md>), [lstm](<https://devfeed.tech/tags/lstm.md>), [margin](<https://devfeed.tech/tags/margin.md>), [mlp](<https://devfeed.tech/tags/mlp.md>), [policies](<https://devfeed.tech/tags/policies.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>)

### AI overview

In a custom Gymnasium inventory environment with hidden two-state regime-switching Poisson demand, an LSTM Recurrent PPO agent outperformed an MLP PPO agent and an optimized (s,S) heuristic at lead times of 0, 3, 6, and 10 days, across 30 deterministic episodes per setting. The comparison used matched compute. The LSTM's advantage over the MLP grew with lead time, from parity at zero days to 2.17x at ten days. PCA of its hidden state suggests the representation shifts from tracking the demand regime at short lead times to encoding multi-step pipeline state at longer ones. These results indicate recurrent memory mattered more as the credit-assignment horizon lengthened in this setup; the supplied abstract does not establish performance beyond the described environment and evaluation.

### Source excerpt

arXiv:2611.15945v1 Announce Type: new Abstract: Classical inventory policies such as (s,S) are provably optimal under stationary demand but cannot adapt when demand switches between hidden regimes, and memoryless deep reinforcement learning policies, confined to a fixed observation window, cannot maintain a belief over latent demand structure. We ask whether a recurrent policy offers a measurable advantage over both a memoryless policy and a classical baseline under regime-switching demand, and how that advantage scales with supply-chain lead time. We build a custom Gymnasium environment in which daily demand is generated by a two-state hidden Markov process with Poisson emissions, so that the regime is known at evaluation time but hidden from the agent. An optimised (s,S) heuristic, an MLP PPO agent, and an LSTM Recurrent PPO agent are trained under matched compute and evaluated at lead times of 0, 3, 6, and 10 days over 30 deterministic episodes each. The recurrent policy outperforms both alternatives at every lead time, and its margin over the MLP widens monotonically with lead time, from parity at lead time 0 to 2.17$\times$ at lead time 10. Principal component analysis of the LSTM hidden state explains the widening gap: the representation shifts from encoding a binary regime indicator at short lead times to encoding multi-step pipeline state at long lead times. Recurrent memory becomes the decisive architectural factor as the credit-assignment horizon lengthens.

## Calibrated Weak Supervision for Post-Harvest Burned-Cropland Mapping Under Label Scarcity

DevFeed: [Calibrated Weak Supervision for Post-Harvest Burned-Cropland Mapping Under Label Scarcity](<https://devfeed.tech/articles/calibrated-weak-supervision-for-post-harvest-burned-cropland-mapping-under-label-scarcity-132837.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.05040>)

Author: Raunak Bhagate, Maitri Polisetty, R I Minu

Published: 2026-11-13T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Calibrated uncertainty](<https://devfeed.tech/topics/calibrated-uncertainty.md>), [mlp](<https://devfeed.tech/topics/mlp.md>)

Tags: [assessment](<https://devfeed.tech/tags/assessment.md>), [classifiers](<https://devfeed.tech/tags/classifiers.md>), [cohen](<https://devfeed.tech/tags/cohen.md>), [concatenation](<https://devfeed.tech/tags/concatenation.md>), [correlation](<https://devfeed.tech/tags/correlation.md>), [cross-validation](<https://devfeed.tech/tags/cross-validation.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [deployment](<https://devfeed.tech/tags/deployment.md>), [external](<https://devfeed.tech/tags/external.md>), [features](<https://devfeed.tech/tags/features.md>), [fire](<https://devfeed.tech/tags/fire.md>), [hotspot](<https://devfeed.tech/tags/hotspot.md>), [india](<https://devfeed.tech/tags/india.md>), [lightgbm](<https://devfeed.tech/tags/lightgbm.md>), [linear](<https://devfeed.tech/tags/linear.md>), [mapping](<https://devfeed.tech/tags/mapping.md>), [neural](<https://devfeed.tech/tags/neural.md>), [optical](<https://devfeed.tech/tags/optical.md>), [perceptron](<https://devfeed.tech/tags/perceptron.md>), [pixel](<https://devfeed.tech/tags/pixel.md>), [supervision](<https://devfeed.tech/tags/supervision.md>)

### AI overview

For Punjab burned-cropland mapping under scarce labels, a weak-supervision framework combining Sentinel-2 spectral change, VIIRS fire context, and MODIS MCD64A1 calibration performed better for district-scale burden assessment than for identifying individual burn scars. The best cross-validation configuration was a very-strict pseudo-label recipe with a multilayer perceptron and full optical features; Random Forest, XGBoost, and LightGBM were practically tied. Deployment retained Random Forest with a threshold of 0.60, giving district-level MODIS agreement R² 0.636 and a mapped-to-MODIS burned-area ratio of 1.005, while pixel-level agreement remained modest (F1 0.205, κ 0.093). Higher agreement with pseudo-labels did not prove greater label correctness. Transfer to Haryana was promising, but Punjab cross-year stability was weak; adding Sentinel-1 did not improve the optical baseline. Optical coverage ended before late-season burns, limiting exact scar mapping and stable annual estimates.

### Source excerpt

arXiv:2610.05040v2 Announce Type: replace Abstract: Mapping post-harvest burned cropland is difficult when fires are small and fragmented and reliable labels are scarce. We developed a calibrated weak-supervision framework for Punjab, India, using Sentinel-2 spectral change, VIIRS active-fire context, and MODIS MCD64A1 as a coarse external calibration and agreement reference. Three pseudo-label recipes, five feature representations, and linear, tree-based, boosted, and neural classifiers were evaluated using nested district-held-out cross-validation over three seeds and five folds. NBR and dNBR were excluded from classifier inputs. The best configuration used the very-strict recipe, a multilayer perceptron, and the full optical feature set (mean Cohen's kappa 0.395, AUROC 0.753, F1 0.747, balanced accuracy 0.703); Random Forest, XGBoost, and LightGBM were practically tied. Higher agreement with held-out pseudo-labels did not establish improved label correctness or independent burned-area accuracy. For deployment, a Random Forest with the very-strict recipe and full optical features was retained. An externally calibrated threshold of 0.60 yielded district-level MODIS agreement of R-squared 0.636, a mapped-to-MODIS burned-area ratio of 1.005, and Spearman correlation of 0.779 with district fire counts. Pixel-level MODIS agreement remained modest (F1 0.205, kappa 0.093). Zero-shot transfer to Haryana was promising (mean kappa 0.641), but Punjab cross-year stability was weak, and Sentinel-1/Sentinel-2 feature concatenation did not improve the optical baseline. Optical observations ended before the seasonal fire context, limiting coverage of late burns. The framework supports district-scale burden assessment and hotspot screening, with limited support for exact scar boundaries or temporally stable annual mapping.

## SMIC: Shared Multiscale Input-Reference Conditioning across Transformer Layers for Natural Language Inference

DevFeed: [SMIC: Shared Multiscale Input-Reference Conditioning across Transformer Layers for Natural Language Inference](<https://devfeed.tech/articles/smic-shared-multiscale-input-reference-conditioning-across-transformer-layers-for-natural-language-inference-132214.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.16270>)

Author: Xin Zhang, Victor S. Sheng

Published: 2026-11-13T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [mlp](<https://devfeed.tech/topics/mlp.md>), [llama](<https://devfeed.tech/topics/llama.md>)

Tags: [attention](<https://devfeed.tech/tags/attention.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [decoder](<https://devfeed.tech/tags/decoder.md>), [diagnostics](<https://devfeed.tech/tags/diagnostics.md>), [inference](<https://devfeed.tech/tags/inference.md>), [language](<https://devfeed.tech/tags/language.md>), [llama](<https://devfeed.tech/tags/llama.md>), [mlp](<https://devfeed.tech/tags/mlp.md>), [models](<https://devfeed.tech/tags/models.md>), [natural-language](<https://devfeed.tech/tags/natural-language.md>), [reference](<https://devfeed.tech/tags/reference.md>), [transformer](<https://devfeed.tech/tags/transformer.md>)

### AI overview

SMIC builds one input-dependent multiscale reference, reuses it across Transformer layers, and lets each layer transform and inject it before the attention update. In a token-context Transformer (TCT), the method achieved the highest observed ten-seed mean accuracy among the reported from-scratch comparisons on all three ANLI rounds; ablations also supported its contribution. The study additionally evaluated a real-valued Llama-style plug-in and deletion/insertion diagnostics on pretrained ModernBERT and Llama models with and without SMIC. Under the evaluated perturbation protocols, TCT-SMIC showed comparatively low pooled-attention redistribution and smaller signed accuracy changes. The abstract also reports a finite-sample bound linking average subset-mass redistribution to mean Jensen-Shannon divergence. It provides no numerical accuracy results or details of the bound.

### Source excerpt

arXiv:2611.16270v1 Announce Type: cross Abstract: Natural Language Inference models can be sensitive to localized lexical changes. We introduce Shared Multiscale Input-Reference Conditioning (SMIC), which constructs a single input-dependent multiscale reference and reuses it across Transformer layers. Each layer applies its own multiscale transformation to the shared reference and injects the resulting conditioning feature before the attention update. We instantiate SMIC in a token--context Transformer (TCT) and conduct from-scratch comparisons against decoder-based and Multi-Layer Perceptron (MLP) baselines across all three ANLI rounds. TCT-SMIC attains the highest observed ten-seed mean accuracy on all three ANLI rounds. The ablations further support the contribution of SMIC. We evaluate a real-valued Llama-style plug-in and extend deletion/insertion diagnostics to pretrained ModernBERT encoder and Llama decoder models with and without SMIC. Across the evaluated perturbation protocols, TCT-SMIC exhibits comparatively low pooled-attention redistribution and smaller signed accuracy changes. We further establish a finite-sample bound on average subset-mass redistribution in terms of the mean Jensen--Shannon divergence between the restricted and renormalized attention distributions.

## Probabilistic State Space Enumeration in Neural Network Computation

DevFeed: [Probabilistic State Space Enumeration in Neural Network Computation](<https://devfeed.tech/articles/probabilistic-state-space-enumeration-in-neural-network-computation-132952.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.15062>)

Author: Victor O. K. Li, Yang Han, Jacqueline C. K. Lam

Published: 2026-11-13T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [mlp](<https://devfeed.tech/topics/mlp.md>), [Deep neural networks](<https://devfeed.tech/topics/deep-neural-networks.md>), [Neural Network](<https://devfeed.tech/topics/neural-network.md>), [Transformer](<https://devfeed.tech/topics/transformer.md>), [Algorithms](<https://devfeed.tech/topics/algorithms.md>)

Tags: [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [deep-neural-networks](<https://devfeed.tech/tags/deep-neural-networks.md>), [dimension](<https://devfeed.tech/tags/dimension.md>), [distribution](<https://devfeed.tech/tags/distribution.md>), [enumeration](<https://devfeed.tech/tags/enumeration.md>), [expectation](<https://devfeed.tech/tags/expectation.md>), [framework](<https://devfeed.tech/tags/framework.md>), [gemma](<https://devfeed.tech/tags/gemma.md>), [groups](<https://devfeed.tech/tags/groups.md>), [inclusion](<https://devfeed.tech/tags/inclusion.md>), [inference](<https://devfeed.tech/tags/inference.md>), [input](<https://devfeed.tech/tags/input.md>), [intermediate](<https://devfeed.tech/tags/intermediate.md>), [key](<https://devfeed.tech/tags/key.md>), [layers](<https://devfeed.tech/tags/layers.md>), [llama](<https://devfeed.tech/tags/llama.md>), [local](<https://devfeed.tech/tags/local.md>), [neural](<https://devfeed.tech/tags/neural.md>), [neural-network](<https://devfeed.tech/tags/neural-network.md>), [probabilistic](<https://devfeed.tech/tags/probabilistic.md>), [state](<https://devfeed.tech/tags/state.md>)

### AI overview

The proposed framework represents a Transformer MLP's dense output exactly as an expectation over importance-corrected masked channel-group states. Its NORDER-II algorithm enumerates joint binary states in decreasing probability without building or sorting the exponential state space; truncation gives an anytime approximation controlled by probability mass and a state budget. At B=2048, reported MLP-output accuracy is 92.4% on Llama-3.2-1B and 93.8% on Gemma-2-2B, versus 76.9% and 79.8% for Channel Top-k. Jointly approximating the first, middle, and last MLP layers keeps WikiText-2 perplexity within 0.06% and 0.54% of dense inference, respectively. Compared with expected-count Grouped Top-k, the method reports 82.6%-126.4% relative accuracy gains for 0.18%-0.19% additional local analytic FLOPs.

### Source excerpt

arXiv:2611.15062v1 Announce Type: new Abstract: Deep neural networks typically realize dense computation by propagating information through all intermediate channels, while many approximation methods rank and retain individual computational components. We ask whether dense neural computation can instead be represented exactly as an expectation over a combinatorial state space, and approximated by evaluating only its most probable configurations. We introduce a novel probabilistic state-space framework for Transformer MLPs. The framework partitions the intermediate dimension into channel groups, defines an input-dependent distribution over binary inclusion states, and uses importance-corrected masked outputs such that the full expectation exactly recovers the dense MLP output. The key algorithmic novelty is NORDER-II, which enumerates complete joint states in decreasing probability without constructing or sorting the exponentially large state space. Truncating this ordering yields an anytime approximation governed by cumulative probability mass and a state budget. On Llama-3.2-1B and Gemma-2-2B, NORDER-II at a state budget of B=2048 achieves 92.4% and 93.8% MLP-output approximation accuracy, compared with 76.9% and 79.8% for Channel Top-k. When the first, middle, and last MLP layers are approximated jointly, WikiText-2 perplexity remains within 0.06% and 0.54% of dense inference. Relative to expected-count Grouped Top-k, NORDER-II yields 82.6% to 126.4% relative accuracy gains with only 0.18% to 0.19% additional local analytic FLOPs. These results establish the significance of joint-state structure beyond atomic salience and point to a new paradigm for neural computation: dense layers can be viewed as structured probabilistic inference over combinatorial computational states. This framework opens a path toward adaptive, probability-mass-controlled computation in pretrained neural networks.

## A Convex-Analytic Reformulation for Poisoning Attacks on Logistic Regression

DevFeed: [A Convex-Analytic Reformulation for Poisoning Attacks on Logistic Regression](<https://devfeed.tech/articles/a-convex-analytic-reformulation-for-poisoning-attacks-on-logistic-regression-133182.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.17143>)

Author: Carina Moreira Costa, Maria Eduarda Pinheiro, Martin Schmidt

Published: 2026-11-13T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Adversarial attacks](<https://devfeed.tech/topics/adversarial-attacks.md>), [mlp](<https://devfeed.tech/topics/mlp.md>)

Tags: [attacks](<https://devfeed.tech/tags/attacks.md>), [classification](<https://devfeed.tech/tags/classification.md>), [convex](<https://devfeed.tech/tags/convex.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [logistic-regression](<https://devfeed.tech/tags/logistic-regression.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [math-oc](<https://devfeed.tech/tags/math-oc.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [regression](<https://devfeed.tech/tags/regression.md>)

### AI overview

The paper reformulates poisoning attacks on ℓ₂-regularized logistic regression, where an attacker selects features for a fixed number of injected examples with prescribed labels to increase loss on clean training data. Strong convexity gives an exact first-order reformulation; softplus convex conjugacy replaces sigmoid relations with a convex optimality-gap inequality, and a change of variables makes stationarity affine. The resulting formulation has bounded bilinear equalities as its only nonconvex constraints, with finite variable bounds tightened through convex-optimization preprocessing. On eight binary classification datasets, local-optimization attacks reduced predictive performance on most datasets, while the convex-analytic formulation produced substantially stronger reported dual bounds than the compared reformulation in SCIP under a fixed time limit.

### Source excerpt

arXiv:2611.17143v1 Announce Type: new Abstract: We study data poisoning attacks on logistic regression with $\ell_2$-regularization, where an attacker chooses the features of a fixed number of injected observations with prescribed labels to maximize the logistic loss on the clean training data. Strong convexity of the defender's problem yields an exact reformulation through its first-order optimality conditions. Using convex conjugacy of the softplus function, we replace the sigmoid relations by a single convex optimality-gap inequality and express the attacker's objective as a concave function. A change of variables makes the stationarity condition affine, leaving bounded bilinear equalities as the only nonconvex constraints. We derive finite bounds on all variables involved in these equalities, including a global bound on the coefficient norm and tighter bounds obtained by preprocessing based on convex optimization. Experiments on eight binary classification datasets evaluate attack effectiveness and compare the two reformulations when solved with \textsf{SCIP}. Attacks obtained by local optimization degrade predictive performance on most datasets, while the convex-analytic formulation produces substantially stronger reported dual bounds within a fixed time limit.

## Edge-Gradient Augmentation under Reinforcement Learning for Temporal Graph

DevFeed: [Edge-Gradient Augmentation under Reinforcement Learning for Temporal Graph](<https://devfeed.tech/articles/edge-gradient-augmentation-under-reinforcement-learning-for-temporal-graph-132270.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.16678>)

Author: Yongqing Zhou, Jiawei Chen, Wenjun Wang, Xinyu Wang, Wei Yu, Qiyao Peng

Published: 2026-11-13T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [data augmentation](<https://devfeed.tech/topics/data-augmentation.md>), [mlp](<https://devfeed.tech/topics/mlp.md>)

Tags: [ablation](<https://devfeed.tech/tags/ablation.md>), [abstract](<https://devfeed.tech/tags/abstract.md>), [adaptive](<https://devfeed.tech/tags/adaptive.md>), [analysis](<https://devfeed.tech/tags/analysis.md>), [area](<https://devfeed.tech/tags/area.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [based](<https://devfeed.tech/tags/based.md>), [can](<https://devfeed.tech/tags/can.md>), [change](<https://devfeed.tech/tags/change.md>), [channel](<https://devfeed.tech/tags/channel.md>), [critical](<https://devfeed.tech/tags/critical.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [data](<https://devfeed.tech/tags/data.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [edge](<https://devfeed.tech/tags/edge.md>), [gradient](<https://devfeed.tech/tags/gradient.md>), [graph](<https://devfeed.tech/tags/graph.md>), [learning](<https://devfeed.tech/tags/learning.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [temporal](<https://devfeed.tech/tags/temporal.md>), [under](<https://devfeed.tech/tags/under.md>)

### AI overview

EG-RL targets temporal link prediction by explicitly encoding edge births and deaths, then using reinforcement learning to select and augment informative edge snapshots. Its policy receives critic-based feedback to optimize prediction accuracy. The architecture combines node and link encoding, token and channel mixing, global pooling, and a two-layer MLP classifier. The abstract reports gains in AUC and average precision across multiple dynamic graph datasets; ablations support the adaptive augmentation and encoding mechanisms. The excerpt gives no dataset names, numerical results, or comparison details.

### Source excerpt

arXiv:2611.16678v1 Announce Type: cross Abstract: Temporal link prediction is critical for understand- ing dynamic interactions in evolving networks, informing appli- cations ranging from social network analysis to recommendation systems. Existing methods typically rely on raw temporal graph data, often neglecting explicit edge-change information that can significantly improve predictive performance. To address this gap, we propose edge-gradient reinforced learning (EG-RL), a framework that explicitly encodes edge-gradient events (edge births and deaths) through dedicated node and link encoding modules. EG-RL uses a policy-driven reinforcement-learning mechanism to dynamically select and augment the most infor- mative edge snapshots, guided by critic-based feedback to opti- mize prediction accuracy. The model comprises distinct modules for node encoding, link encoding, token mixing, and channel mixing, integrated through global pooling and followed by a two-layer multilayer perceptron classifier. Extensive experiments on multiple dynamic graph datasets demonstrate substantial gains in area under the receiver operating characteristic curve and average precision. Ablation studies further confirm the necessity of adaptive edge-gradient augmentation and validate the effectiveness of the proposed encoding and reinforcement- learning mechanisms.

## Sharp Covering-Number Bounds for Feedforward ReLU Networks via Parameter Symmetries

DevFeed: [Sharp Covering-Number Bounds for Feedforward ReLU Networks via Parameter Symmetries](<https://devfeed.tech/articles/sharp-covering-number-bounds-for-feedforward-relu-networks-via-parameter-symmetries-132983.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.15300>)

Author: Ruizhi Deng, Guohao Shen, Tangrui Xiong

Published: 2026-11-13T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Neural Network](<https://devfeed.tech/topics/neural-network.md>), [mlp](<https://devfeed.tech/topics/mlp.md>)

Tags: [architectures](<https://devfeed.tech/tags/architectures.md>), [capacity](<https://devfeed.tech/tags/capacity.md>), [complexity](<https://devfeed.tech/tags/complexity.md>), [continuous](<https://devfeed.tech/tags/continuous.md>), [covering](<https://devfeed.tech/tags/covering.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [fully-connected](<https://devfeed.tech/tags/fully-connected.md>), [function](<https://devfeed.tech/tags/function.md>), [function-approximation](<https://devfeed.tech/tags/function-approximation.md>), [guarantees](<https://devfeed.tech/tags/guarantees.md>), [math-st](<https://devfeed.tech/tags/math-st.md>), [networks](<https://devfeed.tech/tags/networks.md>), [parameter](<https://devfeed.tech/tags/parameter.md>), [permutation](<https://devfeed.tech/tags/permutation.md>), [relu](<https://devfeed.tech/tags/relu.md>), [risk](<https://devfeed.tech/tags/risk.md>), [scaling](<https://devfeed.tech/tags/scaling.md>), [sharp](<https://devfeed.tech/tags/sharp.md>), [space](<https://devfeed.tech/tags/space.md>), [stat-ml](<https://devfeed.tech/tags/stat-ml.md>), [stat-th](<https://devfeed.tech/tags/stat-th.md>), [symmetry](<https://devfeed.tech/tags/symmetry.md>), [weight](<https://devfeed.tech/tags/weight.md>)

### AI overview

The paper argues that parameter counts overstate functional complexity when different network parameters implement the same function. For ReLU and Leaky ReLU networks across fully connected, convolutional, and residual architectures, it characterizes every function-preserving quasianalytic parameter map as a composition of positive channel scaling and channel permutation. Quotienting out these symmetries yields covering-number bounds with exponent reduced from S to S−H and a divisor G for channel permutations. Matching lower bounds establish that the exponent is sharp generally, while the permutation divisor is attained for one-hidden-layer and generically identifiable networks. The work extends the results to continuous piecewise-linear activations and applies the bounds to ERM risk guarantees and capacity control beyond i.i.d. settings.

### Source excerpt

arXiv:2611.15300v1 Announce Type: new Abstract: In neural networks, distinct parameter configurations often realize the same input--output mapping, indicating that nominal parameter counts can overstate functional complexity. In this paper, we characterize the parameter-space symmetries of ReLU and Leaky ReLU networks across fully connected, convolutional, and residual architectures. We prove that every quasianalytic parameter map preserving the network function is a composition of positive channel scaling and channel permutation. By quotienting out both symmetries, we derive uniform covering-number upper bounds in which positive scaling reduces the complexity exponent from the total parameter count $S$ to $S-H$ (with $H$ denoting the number of scalable channels) and permutation symmetry divides the bound by the factorial count $G$ of channel permutations. We establish matching lower bounds showing that the exponent $S-H$ is sharp in general and that the divisor $G$ is attained for one-hidden-layer and generically identifiable networks. We further extend these results to general continuous piecewise-linear activated networks. Finally, we apply our covering-number bounds to derive statistical risk guarantees for empirical risk minimization and capacity control beyond i.i.d.\ settings, and show how the underlying symmetries help explain weight rebalancing in function approximation and flat directions in empirical risk optimization.

## Taming Neural Fields with a Frequency-Lipschitz Curriculum for Sparse-View CT Reconstruction

DevFeed: [Taming Neural Fields with a Frequency-Lipschitz Curriculum for Sparse-View CT Reconstruction](<https://devfeed.tech/articles/taming-neural-fields-with-a-frequency-lipschitz-curriculum-for-sparse-view-ct-reconstruction-133020.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.15652>)

Author: Songtao Hu, Jingrui Wu, Hong Ding, Fanhao Zhu, Yunhao Liu, Suyang Xi

Published: 2026-11-13T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Image processing](<https://devfeed.tech/topics/image-processing.md>), [mlp](<https://devfeed.tech/topics/mlp.md>), [fourier transform](<https://devfeed.tech/topics/fourier-transform.md>)

Tags: [adaptive](<https://devfeed.tech/tags/adaptive.md>), [capacity](<https://devfeed.tech/tags/capacity.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [curriculum](<https://devfeed.tech/tags/curriculum.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [details](<https://devfeed.tech/tags/details.md>), [encoding](<https://devfeed.tech/tags/encoding.md>), [mlp](<https://devfeed.tech/tags/mlp.md>), [neural](<https://devfeed.tech/tags/neural.md>), [noise](<https://devfeed.tech/tags/noise.md>), [performance](<https://devfeed.tech/tags/performance.md>), [projection](<https://devfeed.tech/tags/projection.md>), [quality](<https://devfeed.tech/tags/quality.md>), [radiation](<https://devfeed.tech/tags/radiation.md>), [random](<https://devfeed.tech/tags/random.md>), [sampling](<https://devfeed.tech/tags/sampling.md>), [stable](<https://devfeed.tech/tags/stable.md>), [stage](<https://devfeed.tech/tags/stage.md>), [statistics](<https://devfeed.tech/tags/statistics.md>), [training](<https://devfeed.tech/tags/training.md>), [under](<https://devfeed.tech/tags/under.md>)

### AI overview

FLIC controls an implicit neural representation's capacity during sparse-view CT reconstruction with a coarse-to-fine frequency-Lipschitz curriculum. It combines random Fourier feature encoding with a spectrally normalized MLP under scheduled, non-uniform Lipschitz budgets; the final capacity is calibrated from filtered back-projection gradient statistics, avoiding per-scan manual tuning of that budget. Prior-guided adaptive ray sampling places more integration samples near high-gradient anatomical boundaries. The method uses a single training stage without iterative denoising. The abstract reports state-of-the-art unsupervised reconstruction quality across four public CT datasets in noise-free and low-dose settings; it does not provide quantitative results or comparisons in the supplied excerpt.

### Source excerpt

arXiv:2611.15652v1 Announce Type: cross Abstract: Sparse-view computed tomography (SVCT) reduces radiation dose by acquiring fewer projection views, but the resulting under-determined inverse problem requires careful regularization to prevent reconstruction algorithms from fitting streak artifacts and hallucinated structures. Existing implicit neural representation (INR) methods rely on implicit spectral bias or iterative external denoising, yet they provide limited control over when high-frequency capacity is exposed during optimization. To address this limitation, we propose FLIC (Frequency-Lipschitz INR Curriculum), a capacity-controlled self-prior INR framework that makes the network's representational capacity explicit and schedulable. The core of our method is a joint frequency-Lipschitz curriculum that couples random Fourier feature encoding with spectrally normalized MLP decoding under non-uniform Lipschitz budgets, guiding reconstruction along a coarse-to-fine trajectory from stable global anatomy to fine details. The terminal capacity is automatically calibrated from FBP gradient statistics, enabling scan-adaptive regularization without per-scan manual tuning of the terminal Lipschitz budget. We further introduce a prior-guided adaptive ray sampling strategy that concentrates integration samples near high-gradient anatomical boundaries, improving forward-model fidelity. The entire framework operates in a single training stage without iterative denoising, enabling efficient per-image reconstruction. Extensive experiments on four public CT datasets under noise-free and low-dose settings show that our method achieves state-of-the-art unsupervised reconstruction quality with consistent performance across diverse anatomical structures.

## Robustness Within a Fairness Budget: Parity Frontiers and Equal-Opportunity Reversal

DevFeed: [Robustness Within a Fairness Budget: Parity Frontiers and Equal-Opportunity Reversal](<https://devfeed.tech/articles/robustness-within-a-fairness-budget-parity-frontiers-and-equal-opportunity-reversal-133067.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.16031>)

Author: Pablo Rivas, Jeremy Frank

Published: 2026-11-13T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [responsible-ai](<https://devfeed.tech/topics/responsible-ai.md>), [mlp](<https://devfeed.tech/topics/mlp.md>), [Adversarial attacks](<https://devfeed.tech/topics/adversarial-attacks.md>)

Tags: [adversarial](<https://devfeed.tech/tags/adversarial.md>), [attack](<https://devfeed.tech/tags/attack.md>), [attack-paths](<https://devfeed.tech/tags/attack-paths.md>), [audit](<https://devfeed.tech/tags/audit.md>), [auditing](<https://devfeed.tech/tags/auditing.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [budget](<https://devfeed.tech/tags/budget.md>), [classification](<https://devfeed.tech/tags/classification.md>), [cs-cy](<https://devfeed.tech/tags/cs-cy.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [fairness](<https://devfeed.tech/tags/fairness.md>), [gradient](<https://devfeed.tech/tags/gradient.md>), [gradient-descent](<https://devfeed.tech/tags/gradient-descent.md>), [logistic-regression](<https://devfeed.tech/tags/logistic-regression.md>), [metric](<https://devfeed.tech/tags/metric.md>), [mlp](<https://devfeed.tech/tags/mlp.md>), [perceptron](<https://devfeed.tech/tags/perceptron.md>), [performance](<https://devfeed.tech/tags/performance.md>), [prediction](<https://devfeed.tech/tags/prediction.md>), [regression](<https://devfeed.tech/tags/regression.md>), [risk](<https://devfeed.tech/tags/risk.md>), [robustness](<https://devfeed.tech/tags/robustness.md>)

### AI overview

The study combines PGD adversarial-risk minimization with a squared soft demographic-parity budget, optimized through projected multiplier and one-sided penalty updates. On Adult Income, with sex used for group auditing but excluded from prediction, logistic regression and a one-hidden-layer MLP were tested across three paired seeds, four attack strengths, and five budgets. At ε=0.3, PGD improved balanced accuracy over ERM by 7.77 and 11.45 percentage points and removed 85.7% and 89.5% of attack-induced growth in hard demographic-parity difference. At τ=0.015, constrained training reduced that difference versus PGD by 52.2% and 49.2%, while staying 3.18 and 8.34 points above ERM in balanced accuracy. The sampled budget paths also reversed mean signed equal-opportunity difference; the authors report a rate theorem for the required true-positive-rate shift and say measured group rates satisfy it. The findings are limited to this controlled feature-space study, and the authors caution that one fairness metric cannot replace a full group-rate audit.

### Source excerpt

arXiv:2611.16031v1 Announce Type: new Abstract: Adversarial training protects predictive performance under attack but does not determine how decisions are distributed across protected groups. We formulate binary classification as projected gradient descent (PGD) adversarial-risk minimization under a squared soft demographic-parity (DP) budget, optimized with projected multiplier and one-sided penalty updates. On Adult Income, using the binary sex field supplied by the benchmark for group auditing but excluding it from prediction, we test logistic regression and a one-hidden-layer multilayer perceptron (MLP) across three paired seeds, four PGD strengths, and five budgets. At $\varepsilon=0.3$, PGD training improves balanced accuracy over empirical-risk minimization (ERM) by $7.77$ and $11.45$ percentage points and removes $85.7\%$ and $89.5\%$ of attack-induced growth in hard demographic-parity difference (DPD). At the representative interior value $\tau=0.015$, constrained training cuts DPD relative to PGD training by $52.2\%$ and $49.2\%$, while remaining $3.18$ and $8.34$ balanced-accuracy points above ERM. Across all eight seed-averaged model-attack paths, the sampled budget path strictly orders DPD and accuracy and reverses mean signed equal-opportunity difference (EOpp). A rate theorem gives the exact true-positive-rate shift required for reversal, and the measured group rates satisfy it. Within this controlled feature-space study, robustness and parity admit useful joint operating points, but one fairness metric cannot replace a full group-rate audit.

## PISim: A Unified Photonic Inference Simulator for Hardware-Aware Cross-Layer Design-Space Exploration

DevFeed: [PISim: A Unified Photonic Inference Simulator for Hardware-Aware Cross-Layer Design-Space Exploration](<https://devfeed.tech/articles/pisim-a-unified-photonic-inference-simulator-for-hardware-aware-cross-layer-design-space-exploration-132250.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.16522>)

Author: Andreas Falkenberg

Published: 2026-11-13T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [quantization](<https://devfeed.tech/topics/quantization.md>), [mlp](<https://devfeed.tech/topics/mlp.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [accelerators](<https://devfeed.tech/tags/accelerators.md>), [algebra](<https://devfeed.tech/tags/algebra.md>), [analog](<https://devfeed.tech/tags/analog.md>), [application](<https://devfeed.tech/tags/application.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [architectures](<https://devfeed.tech/tags/architectures.md>), [array](<https://devfeed.tech/tags/array.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [automation](<https://devfeed.tech/tags/automation.md>), [cross](<https://devfeed.tech/tags/cross.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [design](<https://devfeed.tech/tags/design.md>), [design-space-exploration](<https://devfeed.tech/tags/design-space-exploration.md>), [exploration](<https://devfeed.tech/tags/exploration.md>), [hardware](<https://devfeed.tech/tags/hardware.md>), [inference](<https://devfeed.tech/tags/inference.md>), [layer](<https://devfeed.tech/tags/layer.md>), [layer-design](<https://devfeed.tech/tags/layer-design.md>), [physics-optics](<https://devfeed.tech/tags/physics-optics.md>), [simulator](<https://devfeed.tech/tags/simulator.md>), [space](<https://devfeed.tech/tags/space.md>)

### AI overview

PISim combines pretrained neural network execution with physics-based models of photonic accelerator architectures, modulators, optical sources, and readout schemes. It applies stage-resolved models of quantization, noise, thermal drift, loss, and crosstalk, plus cycle-level interface timing, to explore accuracy and throughput before fabrication. In simulated MNIST MLP benchmarks, it found precision operating points peaking at 6 bits for Mach-Zehnder and microring modulators, a 52.5 percentage-point accuracy gap between VCSEL-array and parallel-fanout laser designs attributed under the modeled conditions to power partitioning and correlated source noise, and interface-limited throughput scaling across emitter arrays of 32-128 channels. These results demonstrate the framework's use for accuracy-constrained design-space exploration; the supplied evidence does not establish performance beyond the modeled scenarios.

### Source excerpt

arXiv:2611.16522v1 Announce Type: cross Abstract: Photonic neural accelerators offer ultra-low latency linear algebra, yet their usable numerical precision, sustainable throughput, and application accuracy remain tightly bounded by surrounding electro-optical interfaces and physical non-idealities. Pre-fabrication design-space exploration (DSE) requires a cross-layer electronic design automation (EDA) framework capable of evaluating disparate optical topologies under identical neural workloads and physical assumptions. We present \textbf{PISim}, a unified Photonic Inference Simulator that couples pretrained deep neural network execution with physics-based models of six accelerator architecture families, six electro-optic modulator technologies, four optical source organizations, and direct or coherent readout schemes. PISim embeds stage-resolved impairment operators---covering control and analog-to-digital quantization, finite extinction ratio, shot noise, correlated relative-intensity noise (RIN), thermal drift, propagation loss, and channel crosstalk---alongside an analytical cycle-level interface timing model to expose electronic-optical bottlenecks. Using simulated MNIST multilayer perceptron (MLP) benchmarks, we demonstrate how PISim identifies optimal precision operating points under multi-impairment stress (peaking at 6 bits for Mach--Zehnder and microring modulators), quantifies a 52.5 percentage-point accuracy gap between VCSEL-array and parallel-fanout laser architectures arising, under the modeled conditions, from power partitioning and correlated source noise, and models interface-limited throughput scaling across emitter-array widths from 32 to 128 channels. This comprehensive framework establishes an extensible, reproducible foundation for accuracy-constrained Pareto optimization prior to photonic tape-out.

## CP-MLPs: A Tensor-Rank Theory of Tied and Untied MLP Blocks

DevFeed: [CP-MLPs: A Tensor-Rank Theory of Tied and Untied MLP Blocks](<https://devfeed.tech/articles/cp-mlps-a-tensor-rank-theory-of-tied-and-untied-mlp-blocks-128745.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.13647>)

Author: Md Rifat Arefin, Farzaneh Heidari, Irina Rish, Guillaume Rabusseau

Published: 2026-11-11T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [mlp](<https://devfeed.tech/topics/mlp.md>), [PyTorch](<https://devfeed.tech/topics/pytorch.md>)

Tags: [approximation](<https://devfeed.tech/tags/approximation.md>), [blocks](<https://devfeed.tech/tags/blocks.md>), [border](<https://devfeed.tech/tags/border.md>), [budget](<https://devfeed.tech/tags/budget.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [mlp](<https://devfeed.tech/tags/mlp.md>), [rank](<https://devfeed.tech/tags/rank.md>), [sharing](<https://devfeed.tech/tags/sharing.md>), [stat-ml](<https://devfeed.tech/tags/stat-ml.md>), [theory](<https://devfeed.tech/tags/theory.md>)

### AI overview

CP-MLPs models each multiplicative unit as a rank-1 interaction that detects an input feature, reads a value, and writes to the output. In tied gated blocks such as ReLU², detector and value directions must match; untied blocks such as ReGLU and SwiGLU can separate them. Exact tensor-rank results show untied units use less width for indefinite quadratic interactions and role-asymmetric monomials, while tied units suit symmetric pure-power targets, so the advantage depends on the target. Experimental loss plateaus below exact-representation width do not by themselves prove a positive approximation gap, which is governed by border rank. In selected trained-model comparisons with matched input-projection budget, untied blocks retain an advantage.

### Source excerpt

arXiv:2611.13647v1 Announce Type: new Abstract: Transformer feed-forward blocks combine several design choices: squared activations, gating, and tied versus untied parameter sharing, whose separate contributions to expressivity remain poorly understood. We introduce CP-MLPs, a tensor framework in which each multiplicative unit contributes a rank-1 interaction with three roles: detecting an input feature, reading a value, and writing to the output. Tied gated blocks such as ReLU$^2$ force the detector and value roles to share the same direction; untied gated blocks such as ReGLU and SwiGLU let them differ. We show that this distinction yields exact, degree-wise tensor-rank separations: untied detector-value units are strictly more width-efficient for indefinite quadratic interactions and role-asymmetric monomials, while tied units are structurally matched to symmetric pure-power targets. The width advantage of untying therefore depends on the target structure. Experiments show loss plateaus below the exact-representation width, but these plateaus alone do not establish a positive approximation gap: arbitrarily accurate approximation is governed by border rank. In trained gated MLPs, untied blocks retain an advantage in selected comparisons at matched input-projection budget. Code is available at https://github.com/rarefin/CP-MLPs.

## FUTON: Fourier Tensor Networks for Implicit Neural Representations

DevFeed: [FUTON: Fourier Tensor Networks for Implicit Neural Representations](<https://devfeed.tech/articles/futon-fourier-tensor-networks-for-implicit-neural-representations-128504.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2602.13414>)

Author: Pooya Ashtari, Pourya Behmandpoor, Nikos Deligiannis, Aleksandra Pizurica

Published: 2026-11-11T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Neural Network](<https://devfeed.tech/topics/neural-network.md>), [fourier analysis](<https://devfeed.tech/topics/fourier-analysis.md>), [mlp](<https://devfeed.tech/topics/mlp.md>), [Image](<https://devfeed.tech/topics/image.md>)

Tags: [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [eess-iv](<https://devfeed.tech/tags/eess-iv.md>), [eess-sp](<https://devfeed.tech/tags/eess-sp.md>), [electrical-engineering](<https://devfeed.tech/tags/electrical-engineering.md>), [error](<https://devfeed.tech/tags/error.md>), [image](<https://devfeed.tech/tags/image.md>), [inductive](<https://devfeed.tech/tags/inductive.md>), [linear](<https://devfeed.tech/tags/linear.md>), [memory](<https://devfeed.tech/tags/memory.md>), [mlp](<https://devfeed.tech/tags/mlp.md>), [neural](<https://devfeed.tech/tags/neural.md>), [novel](<https://devfeed.tech/tags/novel.md>), [rank](<https://devfeed.tech/tags/rank.md>), [series](<https://devfeed.tech/tags/series.md>), [signal](<https://devfeed.tech/tags/signal.md>), [size](<https://devfeed.tech/tags/size.md>), [structure](<https://devfeed.tech/tags/structure.md>), [synthesis](<https://devfeed.tech/tags/synthesis.md>), [training](<https://devfeed.tech/tags/training.md>), [view](<https://devfeed.tech/tags/view.md>), [volumes](<https://devfeed.tech/tags/volumes.md>)

### AI overview

FUTON represents signals as generalized Fourier series with coefficients encoded by a low-rank tensor decomposition and read out through a linear or MLP head. Fourier bases capture smoothness and periodicity, while the low-rank structure constrains spectral complexity. For a linear head, the abstract reports an exact error decomposition into series truncation and coefficient low-rank approximation, with bounds estimated from samples before training and a model that attains the upper bound. In matched-size comparisons with twelve implicit neural representations on image fitting, occupancy volumes, and novel view synthesis, FUTON had the best average quality, reached each task's strongest baseline's final quality 4-34 times sooner, inferred fastest, and used the least training memory.

### Source excerpt

arXiv:2602.13414v2 Announce Type: replace-cross Abstract: Implicit neural representations (INRs) have emerged as powerful tools for encoding signals, yet dominant MLP-based designs often suffer from slow convergence, and how their accuracy depends on their size is known only empirically. We introduce FUTON (Fourier Tensor Network), which models signals as generalized Fourier series whose coefficients are parameterized by a low-rank tensor decomposition, read out by a linear or MLP head. FUTON combines complementary inductive biases: Fourier bases capture smoothness and periodicity, while the low-rank parameterization enforces low-dimensional spectral structure. Beyond universal approximation, we show that with a linear head, the best FUTON's error splits exactly into the truncation of the signal's series and the best low-rank error of its coefficients, which we bracket from the samples before training, with a FUTON that attains the upper bound. On image fitting, occupancy volumes, and novel view synthesis, against twelve INRs of matched size, FUTON achieves the best average quality, reaches the final quality of each task's strongest baseline 4-34x sooner than that baseline, infers fastest, and trains in the least memory.

## Topological Dynamics Under Non-Stationary Supervision: Persistent Homology of Neural Network Weight Evolution

DevFeed: [Topological Dynamics Under Non-Stationary Supervision: Persistent Homology of Neural Network Weight Evolution](<https://devfeed.tech/articles/topological-dynamics-under-non-stationary-supervision-persistent-homology-of-neural-network-weight-evolution-128345.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.13701>)

Author: Mitra Varun

Published: 2026-11-11T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [mlp](<https://devfeed.tech/topics/mlp.md>), [Neural Network](<https://devfeed.tech/topics/neural-network.md>)

Tags: [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [fashion](<https://devfeed.tech/tags/fashion.md>), [neural-network](<https://devfeed.tech/tags/neural-network.md>), [persistent-homology](<https://devfeed.tech/tags/persistent-homology.md>), [supervision](<https://devfeed.tech/tags/supervision.md>), [topology](<https://devfeed.tech/tags/topology.md>), [weight](<https://devfeed.tech/tags/weight.md>)

### AI overview

The study finds no demonstrated incremental predictive value from persistent topology for distinguishing how a Fashion-MNIST multilayer perceptron was supervised. Across five paired seeds, models trained with correct labels reached 89.39% mean test accuracy; models trained on a fixed random label permutation fit 67.94% of their assigned targets but did not generalise to the original task. Correct-label models showed descriptively greater time-averaged normalised fc3 H1 persistence, but configured comparisons were not conventionally significant after Holm correction. Early condition classification reached 60.0% balanced accuracy with topology alone, 66.7% with conventional diagnostics, and 60.0% with both. The findings are limited to one architecture and dataset, five seeds, capped H1 distance comparisons, and deferred bottleneck analysis.

### Source excerpt

arXiv:2611.13701v1 Announce Type: new Abstract: We investigate how the persistent topology of neuron- weight point clouds evolves under stable and non-stationary supervision. An identical multilayer perceptron is trained on Fashion-MNIST with correct labels, a fixed random permutation of the training-label vector, or a fresh permutation every epoch. Five paired seeds share initialisation and minibatch order, yielding 15 models trained for 100 epochs. Raw and row-normalised incoming weight clouds are analysed with H0 and H1 Vietoris Rips persistence at 22 checkpoints. Correct-label models reach 89.39% mean true-label test accuracy; fixed-random models fit 67.94% of their assigned training targets without generalising to the original task. Correct-label models have descriptively greater time-averaged normalised fc3 H1 persistence than either random-label condition, but the configured exact comparisons do not reach conventional significance after Holm correction. Early condition classification achieves 60.0% mean balanced accuracy using topology, 66.7% using conventional diagnostics, and 60.0% using both. Thus, no incremental predictive improvement is demonstrated in this setting. The archive contains all 3,300 configured topology cases. Interpretation is limited by five seeds, a single architecture and dataset, 282 capped H1 distance comparisons, and deferred bottleneck analysis.

[Next page](<https://devfeed.tech/topics/mlp.md?cursor=WyIyMDI2LTExLTExVDA1OjAwOjAwKzAwOjAwIiwgIjNmZjdlYzAwLWI0OTMtNDRjZi1iNGQzLWQ3NzFjY2Y0MWI2NyJd>)