# Mamba

A state-space model architecture for linear-time sequence modeling with selective state spaces.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## MambaLIFT: Reference-Based Mechanism-Aware Attribution for VMamba

DevFeed: [MambaLIFT: Reference-Based Mechanism-Aware Attribution for VMamba](<https://devfeed.tech/articles/mambalift-reference-based-mechanism-aware-attribution-for-vmamba-145559.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.25644>)

Author: Jintu Kakati, Niladri B. Puhan

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Mamba](<https://devfeed.tech/topics/mamba.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [attribution](<https://devfeed.tech/tags/attribution.md>), [based](<https://devfeed.tech/tags/based.md>), [block](<https://devfeed.tech/tags/block.md>), [c](<https://devfeed.tech/tags/c.md>), [class](<https://devfeed.tech/tags/class.md>), [composition](<https://devfeed.tech/tags/composition.md>), [contributions](<https://devfeed.tech/tags/contributions.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [difference](<https://devfeed.tech/tags/difference.md>), [eess-iv](<https://devfeed.tech/tags/eess-iv.md>), [effects](<https://devfeed.tech/tags/effects.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [f](<https://devfeed.tech/tags/f.md>), [faithfulness](<https://devfeed.tech/tags/faithfulness.md>), [feature](<https://devfeed.tech/tags/feature.md>), [feature-selection](<https://devfeed.tech/tags/feature-selection.md>), [floating-point](<https://devfeed.tech/tags/floating-point.md>), [mechanism](<https://devfeed.tech/tags/mechanism.md>), [reference](<https://devfeed.tech/tags/reference.md>)

### AI overview

MambaLIFT is a reference-based attribution method for VMamba that traces contributions through selective-scan injection, recurrent memory, LayerNorm, and shared pathways. It decomposes the reference-relative class-logit difference by position, block, scan direction, and mechanism. On ImageNet-1K tests of VMamba-T and VMamba-B, its attributions conserve that difference to floating-point precision. With mean-colour pixel flipping, it scores significantly higher in mean faithfulness than MambaLRP on both models and leads the evaluated methods, including Integrated Gradients with matched references. With blurred replacement, it is non-inferior to MambaLRP on VMamba-T and equivalent on VMamba-B, so the reported advantage depends on the replacement protocol. The authors caution that conservation does not show additive effects under finite interventions: the decomposition does not estimate intervention effects or establish causal validity.

### Source excerpt

arXiv:2611.25644v1 Announce Type: new Abstract: Attributing predictions of visual state-space models remains challenging because input-dependent selective dynamics couple feature selection, recurrent state propagation, and shared computational pathways. We introduce MambaLIFT, a reference-based, multiplier-first attribution framework for VMamba that propagates contributions through its computational graph using interaction-aware rules for selective-scan injection, recurrent memory, LayerNorm, and shared pathways. MambaLIFT decomposes the reference-relative class-logit difference, \(f_c(x)-f_c(x^0)\), by spatial position, VSS block, scan direction, and computational mechanism. Experiments on ImageNet-1K evaluate VMamba-T and VMamba-B, with hyperparameters selected on a calibration set and faithfulness assessed on a disjoint held-out evaluation set. MambaLIFT achieves numerical conservation to floating-point precision across audited attribution interfaces in both configurations. Under mean-colour pixel flipping, it achieves significantly higher mean faithfulness scores than MambaLRP on both configurations and the highest scores among the evaluated methods, including Integrated Gradients with matched references. Under blurred replacement, MambaLIFT is non-inferior to MambaLRP on VMamba-T and equivalent on VMamba-B, indicating that its advantage depends on the replacement protocol. Ablations on VMamba-T show that interaction allocation, temporal propagation, and pathway composition affect attribution maps. Mechanism-level analyses further show that exact relevance conservation does not imply additive effects under finite interventions: the mechanism decomposition accounts for the reference-relative logit difference but does not estimate intervention effects or establish causal validity.

## Channel-Dependent State Space Model for Multivariate Time Series Forecasting

DevFeed: [Channel-Dependent State Space Model for Multivariate Time Series Forecasting](<https://devfeed.tech/articles/channel-dependent-state-space-model-for-multivariate-time-series-forecasting-144942.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2609.36453>)

Author: Yu-Cheng Wu, Fan-Keng Sun, Li-Chun Lu, Duane S. Boning

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Time Series](<https://devfeed.tech/topics/time-series.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Mamba](<https://devfeed.tech/topics/mamba.md>)

Tags: [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [scalability](<https://devfeed.tech/tags/scalability.md>), [state-space-model](<https://devfeed.tech/tags/state-space-model.md>), [time-series-forecasting](<https://devfeed.tech/tags/time-series-forecasting.md>), [variables](<https://devfeed.tech/tags/variables.md>)

### AI overview

Chameleon combines selective state space models with the Kalman filter's measurement-update formulation to model fine-grained, data-dependent interactions across variables, while retaining an SSM backbone for temporal dynamics and scaling linearly with variable count. It uses GatedDeltaNet and stochastic perturbation of reversible instance normalization to support generalization. On strongly dependent ODE and PEMS datasets, it reports the best MSE and MAE across all settings; across 28 standard benchmark settings, it beats each baseline on MSE in at least 27 and on MAE in at least 22. The abstract reports competitive efficiency and favorable scalability on Traffic and ETT. These are reported results from the supplied abstract.

### Source excerpt

arXiv:2609.36453v3 Announce Type: replace Abstract: Multivariate time series forecasting (MTSF) is critical across many real-world domains. Existing deep learning approaches fall into two paradigms with distinct limitations: channel-independent (CI) methods ignore cross-variable dependencies and model only temporal dynamics, while channel-dependent (CD) methods consider both but typically rely on architectural compromises to mitigate overfitting and computational overhead. We therefore propose Chameleon, a CD state space model (SSM) that enables data-dependent, fine-grained interactions across variables while scaling linearly with the number of variables. By connecting selective SSMs with the Kalman filter, we adapt the latter's measurement-update formulation for cross-variable modeling while preserving the SSM backbone for robust temporal modeling. We further identify favorable inductive biases of GatedDeltaNet for time series, adapt it as our backbone, and improve generalization through additional techniques, including a previously unexplored stochastic perturbation of reversible instance normalization. On strongly dependent ODE and PEMS datasets, Chameleon achieves the best MSE and MAE across all settings, while its CI ablation and prior CD methods incur 61-178% higher MSE on average. Across 28 standard benchmark settings, Chameleon also achieves better MSE and MAE than each baseline in at least 27 and 22 cases, respectively. Training-time, FLOP, and peak-memory analyses on Traffic and ETT further demonstrate competitive efficiency and favorable scalability across different variable counts.

## Mamba Flow Matching Neural Processes: Linear-Time Inference for Irregularly Observed Spatial Fields

DevFeed: [Mamba Flow Matching Neural Processes: Linear-Time Inference for Irregularly Observed Spatial Fields](<https://devfeed.tech/articles/mamba-flow-matching-neural-processes-linear-time-inference-for-irregularly-observed-spatial-fields-145168.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.26355>)

Author: Cosmo Santoni, Giovanni Charles, Timothy James Hitge, Oliver Watson, Elizaveta Semenova

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Mamba](<https://devfeed.tech/topics/mamba.md>), [Neural Network](<https://devfeed.tech/topics/neural-network.md>), [Deep neural networks](<https://devfeed.tech/topics/deep-neural-networks.md>)

Tags: [2](<https://devfeed.tech/tags/2.md>), [3](<https://devfeed.tech/tags/3.md>), [abstract](<https://devfeed.tech/tags/abstract.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [attention](<https://devfeed.tech/tags/attention.md>), [based](<https://devfeed.tech/tags/based.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [but](<https://devfeed.tech/tags/but.md>), [combined](<https://devfeed.tech/tags/combined.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [fields](<https://devfeed.tech/tags/fields.md>), [flow](<https://devfeed.tech/tags/flow.md>), [flow-matching](<https://devfeed.tech/tags/flow-matching.md>), [inference](<https://devfeed.tech/tags/inference.md>), [linear](<https://devfeed.tech/tags/linear.md>), [mamba](<https://devfeed.tech/tags/mamba.md>), [matching](<https://devfeed.tech/tags/matching.md>), [neural](<https://devfeed.tech/tags/neural.md>), [processes](<https://devfeed.tech/tags/processes.md>), [spatial](<https://devfeed.tech/tags/spatial.md>), [time](<https://devfeed.tech/tags/time.md>)

### AI overview

MambaFlowNP uses conditional flow matching and a multi-directional Mamba-2 scan to jointly predict irregularly observed spatial fields with linear compute. Context and query tokens share a sequence, and each query reads the scan state at its own position; this avoids reducing context to one hidden summary and makes predictions consistent across query sets. On synthetic Matérn Gaussian-process benchmarks it was competitive with leading attention-based joint-predictive baselines. For NOAA temperature interpolation, it achieved the lowest reported RMSE and CRPS, near-nominal coverage across tiers, and ran 114-784 times faster than the strongest joint-predictive baseline. A forward pass handled one million points in under a second on an NVIDIA L40S, though predictive quality was evaluated only up to 5,580 points.

### Source excerpt

arXiv:2611.26355v1 Announce Type: new Abstract: Interpolating a spatial field, such as daily maximum temperature, from sparse, irregularly located observations recurs across the environmental and health sciences. Standard Gaussian-process kriging refits hyperparameters and solves an $O(N^3)$ system for every new field. When many related fields are observed, such as daily maps from one sensor network, neural processes (NPs) instead learn the interpolation map from past fields and predict a new field in one forward pass. Joint-predictive variants, which sample whole fields rather than independent per-location values, typically use self-attention over context and query points, costing $O(K^2)$ in the combined point count $K$ and limiting practical use to a few thousand points. State-space models offer a linear-time alternative, but a direct state-space adaptation compresses the context into one hidden state before reading queries, reducing what the decoder sees. We propose MambaFlowNP, a linear-compute joint-predictive NP trained with conditional flow matching. A multi-directional Mamba-2 scan reads context and query tokens in one shared sequence, so each query's velocity is read from its own scan position rather than a compressed context summary, and because queries only read the scan state, predictions are consistent across query sets. MambaFlowNP is competitive with the strongest attention-based joint-predictive baselines on synthetic Mat\'ern Gaussian-process benchmarks, and on regional to continental-scale NOAA temperature interpolation it attains the lowest RMSE and CRPS and near-nominal coverage on every tier, while running 114--784$\times$ faster than the strongest joint-predictive baseline. A single forward pass scales to $10^6$ points on one NVIDIA L40S GPU in under a second. This makes joint prediction feasible for continental-scale, irregularly sampled sensor networks; predictive quality is evaluated up to $K=5{,}580$.

## StreamPhy: Streaming Inference of High-Dimensional Physical Dynamics via State Space Models

DevFeed: [StreamPhy: Streaming Inference of High-Dimensional Physical Dynamics via State Space Models](<https://devfeed.tech/articles/streamphy-streaming-inference-of-high-dimensional-physical-dynamics-via-state-space-models-143254.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2605.07384>)

Author: Panqi Chen, Yifan Sun, Shikai Fang, Xiao Fu, Lei Cheng

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [streaming data analytics](<https://devfeed.tech/topics/streaming-data-analytics.md>), [Mamba](<https://devfeed.tech/topics/mamba.md>), [Streaming](<https://devfeed.tech/topics/streaming.md>)

Tags: [adaptive](<https://devfeed.tech/tags/adaptive.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [diffusion](<https://devfeed.tech/tags/diffusion.md>), [encoder](<https://devfeed.tech/tags/encoder.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [evolution](<https://devfeed.tech/tags/evolution.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [faster](<https://devfeed.tech/tags/faster.md>), [field](<https://devfeed.tech/tags/field.md>), [fields](<https://devfeed.tech/tags/fields.md>), [framework](<https://devfeed.tech/tags/framework.md>), [functional](<https://devfeed.tech/tags/functional.md>), [generation](<https://devfeed.tech/tags/generation.md>), [generative](<https://devfeed.tech/tags/generative.md>), [generative-models](<https://devfeed.tech/tags/generative-models.md>), [improvement](<https://devfeed.tech/tags/improvement.md>), [inference](<https://devfeed.tech/tags/inference.md>), [models](<https://devfeed.tech/tags/models.md>), [state-space](<https://devfeed.tech/tags/state-space.md>), [state-space-models](<https://devfeed.tech/tags/state-space-models.md>), [streaming](<https://devfeed.tech/tags/streaming.md>)

### AI overview

StreamPhy estimates evolving physical fields in real time from irregular sparse measurements. It combines an observation-pattern-adaptive encoder, a structured state-space model for memory-efficient updates across irregular time intervals, and an FT-FiLM decoder that generates continuous fields. The authors prove FT-FiLM can represent a richer function class than the functional Tucker model. In experiments on three physical systems with challenging sampling patterns, they report at least 48% better accuracy than state-of-the-art baselines and inference 20-100 times faster than diffusion-based methods. The available text is an abstract and does not specify the systems, evaluation setup, or comparisons behind these results.

### Source excerpt

arXiv:2605.07384v4 Announce Type: replace Abstract: Inferring the evolution of high-dimensional and multi-modal (e.g., spatio-temporal) physical fields from irregular sparse measurements in real time is a fundamental challenge in science and engineering. Existing approaches, including diffusion-based generative models and functional tensor methods, typically operate in offline settings, depend on full temporal observations, or incur substantial inference cost. We propose StreamPhy, an end-to-end framework that enables efficient and accurate streaming inference of full-field physical dynamics from incoming irregular sparse measurements. The framework integrates a data-adaptive observation encoder that is robust to arbitrary observation patterns, a structured state-space model that supports memory-efficient online updates across irregular time intervals, and an expressive Functional Tensor Feature-wise Linear Modulation (FT-FiLM) decoder for continuous-field generation. We prove that FT-FiLM is more expressive than the functional Tucker model, admitting a richer function class for handling complex dynamics. Experiments on three representative physical systems under challenging sampling patterns show that StreamPhy consistently outperforms state-of-the-art baselines, with at least 48\% improvement in accuracy and up to 20--100X faster inference than diffusion-based methods.

## BiMa-ECGSR: reconstructing the 12-lead ECG from reduced lead sets with bidirectional state-space models

DevFeed: [BiMa-ECGSR: reconstructing the 12-lead ECG from reduced lead sets with bidirectional state-space models](<https://devfeed.tech/articles/bima-ecgsr-reconstructing-the-12-lead-ecg-from-reduced-lead-sets-with-bidirectional-state-space-models-142640.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.23345>)

Author: Ugo Lomoio, Pierangelo Veltri, Pietro Hiram Guzzi

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Mamba](<https://devfeed.tech/topics/mamba.md>)

Tags: [classifier](<https://devfeed.tech/tags/classifier.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [diffusion](<https://devfeed.tech/tags/diffusion.md>), [ecg](<https://devfeed.tech/tags/ecg.md>), [encoder-decoder](<https://devfeed.tech/tags/encoder-decoder.md>), [lead](<https://devfeed.tech/tags/lead.md>), [mamba](<https://devfeed.tech/tags/mamba.md>), [regression](<https://devfeed.tech/tags/regression.md>), [state-space](<https://devfeed.tech/tags/state-space.md>), [state-space-models](<https://devfeed.tech/tags/state-space-models.md>)

### AI overview

BiMa-ECGSR reconstructs 12-lead ECGs from six reduced lead sets using a bidirectional Mamba encoder-decoder, with regression (BiMa-SR) or diffusion (DiBiMa-SR). Trained on PTB-XL and evaluated on its test fold and two external cohorts, BiMa-SR outperformed a least-squares linear filter in all 18 input-set/cohort comparisons, though its margin narrowed as chest leads were added. From lead II alone, median correlation across predicted leads was 0.914 versus 0.784 for the filter; regression also exceeded diffusion (0.914 versus 0.903 one-step and 0.862 with six DDIM steps). A classifier trained on real ECGs scored 0.843 AUROC on lead-II reconstructions and 0.908 on reconstructions from I+II+V1+V4, versus 0.912 on real PTB-XL ECGs; external three- or four-lead results were within 0.004 of real ECGs. BiMa-SR has 2.68 million parameters and took 45.8 ms per 10-second record on a GPU. Reconstruction enables use with 12-lead tools but cannot recover information absent from measured leads.

### Source excerpt

arXiv:2611.23345v1 Announce Type: cross Abstract: Wearables and monitors record few electrocardiogram (ECG) leads; most diagnostic tools assume 12 leads. BiMa-ECGSR reconstructs the 12-lead ECG from reduced lead sets with a bidirectional Mamba state-space encoder--decoder, as regression (BiMa-SR) or diffusion (DiBiMa-SR). Models were trained on PTB-XL for six input sets (I; II; I+II; I+II+V1; I+II+V2; I+II+V1+V4), see only the measured leads at inference, and were tested on the PTB-XL test fold (2,198 records) and two external cohorts (Georgia, 10,275 records; Chapman-Shaoxing/Ningbo, 43,559). From lead II alone, the pooled median Pearson correlation (PCC) over the 11 predicted leads was 0.914 on PTB-XL, against 0.784 for a least-squares linear filter (lead I: 0.906 against 0.789). As limb leads are exactly derivable from I and II, multi-lead inputs were compared on predicted chest leads: 0.939 against 0.849 for I+II and 0.979 against 0.962 for I+II+V1+V4. BiMa-SR exceeded the linear comparator in all 18 configuration--cohort pairs (by 0.012--0.136), a margin that shrank as chest leads were added. From lead II, regression was more accurate than diffusion (DiBiMa-SR: 0.903 one-step, 0.862 with six DDIM steps). A 12-lead classifier trained only on real ECGs reached a superclass macro AUROC of 0.912 on real PTB-XL ECGs, 0.843 on reconstructions from lead II and 0.908 from I+II+V1+V4; externally, reconstructions from three or four leads scored within 0.004 of real ECGs. BiMa-SR (2.68\,M parameters) takes 45.8 ms per 10-s record on a GPU. Reconstruction makes reduced-lead recordings usable by 12-lead tools; it cannot add information absent from the measured leads.

## Elastic Selective Spectral Hybrids for Train-Once, Export-Many Budgeted Inference

DevFeed: [Elastic Selective Spectral Hybrids for Train-Once, Export-Many Budgeted Inference](<https://devfeed.tech/articles/elastic-selective-spectral-hybrids-for-train-once-export-many-budgeted-inference-143315.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2609.32486>)

Author: Dachuan Song, Chuchu Chen, Xuan Wang

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Mamba](<https://devfeed.tech/topics/mamba.md>), [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>), [model-deployment](<https://devfeed.tech/topics/model-deployment.md>), [Model Optimizer](<https://devfeed.tech/topics/model-optimizer.md>)

Tags: [2](<https://devfeed.tech/tags/2.md>), [3](<https://devfeed.tech/tags/3.md>), [abstract](<https://devfeed.tech/tags/abstract.md>), [also](<https://devfeed.tech/tags/also.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [attention](<https://devfeed.tech/tags/attention.md>), [b300](<https://devfeed.tech/tags/b300.md>), [batch](<https://devfeed.tech/tags/batch.md>), [but](<https://devfeed.tech/tags/but.md>), [can](<https://devfeed.tech/tags/can.md>), [capacity](<https://devfeed.tech/tags/capacity.md>), [channel](<https://devfeed.tech/tags/channel.md>), [combines](<https://devfeed.tech/tags/combines.md>), [compact-models](<https://devfeed.tech/tags/compact-models.md>), [computing](<https://devfeed.tech/tags/computing.md>), [configuration](<https://devfeed.tech/tags/configuration.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [elastic](<https://devfeed.tech/tags/elastic.md>), [export](<https://devfeed.tech/tags/export.md>), [inference](<https://devfeed.tech/tags/inference.md>), [train](<https://devfeed.tech/tags/train.md>)

### AI overview

ESSH is designed to train one language model and export smaller variants for different compute and latency budgets. It turns each Hankel spectral channel into an independent recurrent unit with input-dependent decay and write/read gates, addressing the fixed retention behavior of linear time-invariant spectral filters while keeping channels truncatable. The model combines these selective mixers with sliding-window attention; joint training uses a two-rate capacity map and full-model distillation to reduce channel count and feed-forward width. The authors report chunked parallel training and fused recurrent decoding that skips discarded channels. Full-capacity language-modeling quality is reported as comparable to similarly sized independently trained models, with a smooth quality-cost trade-off in smaller exports. Evaluations cover language understanding, retrieval, cross-domain text, and DNA. For a 1.53B configuration, batch-one decoding is reported at 1.37 ms per token on a B300, 2.14-2.80x faster than the tested Mamba-2 and Mamba-3 implementations and 3.03x faster than Transformer++ at matched parameter counts.

### Source excerpt

arXiv:2609.32486v2 Announce Type: replace Abstract: Deploying a language model under different computing and latency budgets calls for compact models with different quality-cost trade-offs. Towards this end, elastic spectral state space models provide ordered and temporally decomposed channels that can be truncated, but they are associated with linear time-invariant filters that cannot selectively preserve relevant past information or forget irrelevant information as the context evolves. To address this, we introduce the Elastic Selective Spectral Hybrid (ESSH), which realizes each Hankel spectral channel as an independent recurrent unit using fitted damped rotation modes. It also features an input-dependent decay and write/read gates that make temporal retention and state update input-dependent while preserving channel-wise truncation and a structured recurrence for efficient execution. ESSH combines these selective spectral mixers with sliding-window attention and jointly trains multiple capacities by reducing spectral-channel count and feed-forward width at different rates through a two-rate capacity map with full-model distillation. The resulting models support chunked parallel training and fused recurrent decoding while avoiding computation for discarded channels. At full capacity, ESSH achieves language-modeling quality comparable to similarly sized independently trained models, while smaller exports exhibit a smooth quality-cost trade-off. We validate the effectiveness of the proposed framework using language understanding, retrieval, cross-domain text, and DNA experiments, by assessing quality retention and the trade-off against independently trained and elastic baselines. At the 1.53B model configuration, fused batch-one decoding takes 1.37 ms per token on a B300, providing a 2.14-2.80x speedup over the tested Mamba-2 and Mamba-3 implementations and 3.03x over Transformer++ at matched parameter counts.

## A Log-Timescale Perturbation Bound for Selective State Space Models at Every Sequence Length

DevFeed: [A Log-Timescale Perturbation Bound for Selective State Space Models at Every Sequence Length](<https://devfeed.tech/articles/a-log-timescale-perturbation-bound-for-selective-state-space-models-at-every-sequence-length-141707.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.21952>)

Author: Jack Humphries, Daniel Midgley

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Mamba](<https://devfeed.tech/topics/mamba.md>)

Tags: [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [limit-order-book](<https://devfeed.tech/tags/limit-order-book.md>), [mamba](<https://devfeed.tech/tags/mamba.md>), [scan](<https://devfeed.tech/tags/scan.md>), [state-space-models](<https://devfeed.tech/tags/state-space-models.md>), [streaming](<https://devfeed.tech/tags/streaming.md>), [streaming-data](<https://devfeed.tech/tags/streaming-data.md>)

### AI overview

The paper proves an all-sequence-length perturbation bound for a selective-scan cell by measuring timescale changes on a log scale. The bound shrinks with input-change size and requires no lower bound on the timescale; a construction makes it within about a factor of 2 of best possible for small changes. The authors extend the result to the update in Mamba's released code when timescale is bounded above. In block-level evaluations on two small models trained on one day of order-book data, the existing-analysis bound is tighter for at most the first 8 steps with bounded timescale, and is looser at every length with Mamba's softplus timescale (certified lower bound as small as 10⁻²¹). The bounds cover every block input but exceed changes found by gradient attacks by more than five orders of magnitude.

### Source excerpt

arXiv:2611.21952v1 Announce Type: new Abstract: Mamba models deployed on streaming data, such as limit order book updates, run far past the sequence lengths they were trained on. A guarantee on how much a small change of input can move the hidden state therefore has to hold at every length. The closest existing bound we know of measures the change of the timescale additively, and it grows linearly with the sequence length unless the timescale is bounded away from zero. In this paper, we measure the change on a log scale instead. The key observation is that the decay factor of each step then moves by at most a fixed fraction of that step's input weight, so the per-step errors cannot pile up. Using this, we prove that two runs of a selective-scan cell from a common start stay within a bound that holds at every length, shrinks to zero with the size of the change and needs no lower bound on the timescale. We give a construction showing that, for small changes, this bound is within a factor of about 2 of the best possible, and we extend it to the update used in Mamba's released code when the timescale is bounded above. We then evaluate block-level bounds built from the theorem on two small models trained on one day of order book data. With a bounded timescale, a bound built on the existing analysis is tighter than ours for at most the first 8 steps. With Mamba's softplus timescale, whose certified lower bound is as small as $10^{-21}$, it is looser at every length. Our bounds hold for every input to a block, but they remain more than five orders of magnitude above the changes that gradient attacks find.

## FM-Mamba: An Efficient Native 2D Selective State Space Model with Stability-Aware Discretization

DevFeed: [FM-Mamba: An Efficient Native 2D Selective State Space Model with Stability-Aware Discretization](<https://devfeed.tech/articles/fm-mamba-an-efficient-native-2d-selective-state-space-model-with-stability-aware-discretization-139954.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.22061>)

Author: Shengyuan Hu, Yiyong Sun, Weihang Shan, Yongzhe Chang

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Mamba](<https://devfeed.tech/topics/mamba.md>), [Image processing](<https://devfeed.tech/topics/image-processing.md>)

Tags: [2d](<https://devfeed.tech/tags/2d.md>), [coupling](<https://devfeed.tech/tags/coupling.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-sy](<https://devfeed.tech/tags/cs-sy.md>), [delta](<https://devfeed.tech/tags/delta.md>), [eess-sy](<https://devfeed.tech/tags/eess-sy.md>), [grid](<https://devfeed.tech/tags/grid.md>), [images](<https://devfeed.tech/tags/images.md>), [independent](<https://devfeed.tech/tags/independent.md>), [inference](<https://devfeed.tech/tags/inference.md>), [input](<https://devfeed.tech/tags/input.md>), [local](<https://devfeed.tech/tags/local.md>), [mamba](<https://devfeed.tech/tags/mamba.md>), [model](<https://devfeed.tech/tags/model.md>), [native](<https://devfeed.tech/tags/native.md>), [position](<https://devfeed.tech/tags/position.md>), [projection](<https://devfeed.tech/tags/projection.md>), [ratio](<https://devfeed.tech/tags/ratio.md>), [scan](<https://devfeed.tech/tags/scan.md>), [space](<https://devfeed.tech/tags/space.md>), [spatial](<https://devfeed.tech/tags/spatial.md>), [stable](<https://devfeed.tech/tags/stable.md>), [state](<https://devfeed.tech/tags/state.md>)

### AI overview

FM-Mamba processes 2D features directly on the grid using a coupled Fornasini-Marchesini recurrence, avoiding feature-map flattening. Two position- and direction-dependent step sizes govern propagation along the spatial axes, while cross-axis coupling adds a diagonal-predecessor term. A normalized ratio gives a piecewise stability bound; steps outside it are projected back, and the criterion is checked per position after training to suppress systematic local amplification without global damping. A dual-diagonal wavefront scan gathers from two anti-diagonals, with one input projection per current state. The supplied abstract describes the mechanism but provides no benchmark results or quantitative performance comparisons.

### Source excerpt

arXiv:2611.22061v1 Announce Type: cross Abstract: When applied to two-dimensional features such as images, Mamba-based methods either flatten the feature map into 1D sequences and fuse multiple scans, which breaks spatial adjacency and multiplies computation, or recurse directly on the 2D grid but rescale the recurrence with a fixed input-independent factor and aggregate only the predecessors on a single anti-diagonal, leaving the discrete step unconstrained by 2D stability theory. We propose \textbf{FM-Mamba}, a native 2D selective state space model built on the general Fornasini--Marchesini (FM) model. Its coupled recurrence evolves along both spatial axes; cross-axis coupling induces a discrete diagonal-predecessor term $\tilde A_0$, and two independent input-dependent step sizes $\Delta_1(i,j)$, $\Delta_2(i,j)$ govern propagation along the two axes, so that the coefficients adapt per position and per direction. A normalized ratio $r_k=a_{0,k}/(a_{1,k}a_{2,k})$ yields a closed-form piecewise bound on the two step sizes; when a configuration falls outside the stable interval the steps are projected onto this bound, and the criterion is re-verified per position after training, which suppresses systematic local amplification without global damping. The recurrence is executed by a dual-diagonal wavefront scan: each update gathers information from two anti-diagonals, and the input enters the current state through a single projection, so that neither training nor inference flattens the feature map. Code is available at https://github.com/BitSsm/FM-Mamba-Runtime

## Latent Lattice Language Models

DevFeed: [Latent Lattice Language Models](<https://devfeed.tech/articles/latent-lattice-language-models-137722.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.21608>)

Author: Shivam Duggal, Zongze Wu, William T. Freeman, Antonio Torralba, Richard Zhang, Eli Shechtman

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Language models](<https://devfeed.tech/topics/language-models.md>), [Transformers](<https://devfeed.tech/topics/transformers.md>), [Mamba](<https://devfeed.tech/topics/mamba.md>)

Tags: [attention](<https://devfeed.tech/tags/attention.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [gated-deltanet](<https://devfeed.tech/tags/gated-deltanet.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [lattice](<https://devfeed.tech/tags/lattice.md>), [mamba](<https://devfeed.tech/tags/mamba.md>), [memory](<https://devfeed.tech/tags/memory.md>), [mlp](<https://devfeed.tech/tags/mlp.md>), [parameter](<https://devfeed.tech/tags/parameter.md>), [perplexity](<https://devfeed.tech/tags/perplexity.md>), [recurrent](<https://devfeed.tech/tags/recurrent.md>), [scaling](<https://devfeed.tech/tags/scaling.md>), [sequence](<https://devfeed.tech/tags/sequence.md>), [state](<https://devfeed.tech/tags/state.md>), [stream](<https://devfeed.tech/tags/stream.md>), [time](<https://devfeed.tech/tags/time.md>), [token](<https://devfeed.tech/tags/token.md>), [transformer](<https://devfeed.tech/tags/transformer.md>), [transformers](<https://devfeed.tech/tags/transformers.md>)

### AI overview

The Lattice of Latents adds recurrent per-layer latent streams that persist across tokens alongside the Transformer's depth-wise residual stream. These states directly condition attention, KV-cache writes, MLP computation, and decoding. In reported tests on 20B FineWeb-Edu tokens, a parameter-matched 406M-parameter Lattice Transformer reduced validation perplexity from 15.41 to 14.40. Combined with hyper-connections and input residuals, it reached 13.61, versus 14.22 for a Transformer with those enhancements. Lattice variants also improved perplexity at every tested native state size on Gated DeltaNet, Mamba-2, Triadic GDN, and GDN-Mamba-2 hybrids; Lattice-GDN and Lattice-Triadic-GDN beat pure baselines with four times larger native states. The evidence is limited to the abstract excerpt.

### Source excerpt

arXiv:2611.21608v1 Announce Type: cross Abstract: Language models typically increase per-token computation through deeper or looped layers and latent reasoning steps, while richer residual connections improve information flow across layers. We instead increase computational capacity: the number of active computational states a model maintains. We introduce the Lattice of Latents, a set of per-layer horizontal latent streams that run recurrently along time, alongside the vertical residual stream that flows through depth. Like hyper-connections, mHC, and attention or value residuals, the Lattice enhances the residual stream, but rather than adding or rerouting residual pathways through depth, it provides companion computational states that persist across tokens. Unlike hybrid Transformer-SSM architectures, whose recurrent states primarily serve as internal memory for sequence mixing, these latents directly condition the vertical stream's attention, KV-cache writes, MLP computation, and decoding. On Transformers trained on 20B FineWeb-Edu tokens, a 406M-parameter Lattice model lowers validation perplexity from 15.41 to 14.40 over a parameter-matched Transformer. These gains extend across architectural variants. The Lattice composes with prior residual-stream enhancements: with hyper-connections and input residuals, it lowers validation perplexity from 15.41 (base Transformer) and 14.22 (Transformer with the same enhancements) to 13.61. On Gated DeltaNet, Mamba-2, Triadic GDN, and GDN-Mamba-2 hybrids, it improves perplexity at every tested native state size, with Lattice-GDN and Lattice-Triadic-GDN variants outperforming pure baselines with 4x larger native states. These results identify persistent per-layer state as a complementary axis for scaling language-model computation.

## Certified, Not Explained: Exact Token Erasure in Gated Vision State Space Models

DevFeed: [Certified, Not Explained: Exact Token Erasure in Gated Vision State Space Models](<https://devfeed.tech/articles/certified-not-explained-exact-token-erasure-in-gated-vision-state-space-models-139107.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.20897>)

Author: Sadia Parveen, Tehseen Zia

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Mamba](<https://devfeed.tech/topics/mamba.md>), [object-detection](<https://devfeed.tech/topics/object-detection.md>)

Tags: [attention](<https://devfeed.tech/tags/attention.md>), [audit](<https://devfeed.tech/tags/audit.md>), [bottleneck](<https://devfeed.tech/tags/bottleneck.md>), [convolution](<https://devfeed.tech/tags/convolution.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [gate](<https://devfeed.tech/tags/gate.md>), [information](<https://devfeed.tech/tags/information.md>), [interpretability](<https://devfeed.tech/tags/interpretability.md>), [invariant](<https://devfeed.tech/tags/invariant.md>), [mamba](<https://devfeed.tech/tags/mamba.md>), [maps](<https://devfeed.tech/tags/maps.md>), [mask](<https://devfeed.tech/tags/mask.md>), [oracle](<https://devfeed.tech/tags/oracle.md>), [partitioning](<https://devfeed.tech/tags/partitioning.md>), [state-space-models](<https://devfeed.tech/tags/state-space-models.md>), [vision](<https://devfeed.tech/tags/vision.md>)

### AI overview

In a lung-nodule concept bottleneck model, exact token erasure comes from masking the causal convolution inputs and assigning closed tokens zero readout weight; gating the selective step size Δ alone is neither necessary nor sufficient. Across 15 cross-validated LIDC-IDRI models, the specified masks yielded exactly zero logit change under token knockouts, while violations produced changes of 4.1-6.7. The gated model's pooled out-of-fold AUROC (0.8425 ± 0.0085) matched the ungated bottleneck (0.8428). The authors interpret the gate as a certified support set, not an explanation: post-hoc rankers ordered tokens better, and gate maps from permuted-label training resembled real-label maps. Gate-selected tokens carried more task information than random tokens, but no more than hidden-attention selections. The abstract also reports that partitioning the readout slightly reduced leakage under an earlier recipe; the final model's leakage was at the level implied by correlations among true concepts.

### Source excerpt

arXiv:2611.20897v1 Announce Type: cross Abstract: Vision models built on Mamba are often explained through their selective step size Delta, on the assumption that a token the scan does not admit cannot influence the prediction. We test that in a concept bottleneck model for lung-nodule malignancy built on a single gated, bidirectional selective state-space block. The answer becomes an interpretability framework: a guarantee, a procedure that checks it, and a pre-registered audit of what it explains. For a fixed gate pattern, masking the input of the causal convolution of the block and giving closed tokens zero weight at readout makes the concept and task logits exactly invariant to the closed tokens. Gating Delta is neither necessary nor sufficient. On LIDC-IDRI, a path-by-path knockout over 15 cross-validated models agrees with this in every run: the largest logit change is exactly 0 whenever the conditions hold and between 4.1 and 6.7 when they do not. The gated model reaches a pooled out-of-fold AUROC of 0.8425 +/- 0.0085, level with an ungated Mamba bottleneck (0.8428). The guarantee belongs to the masked model, not to the gate as an explanation. Post-hoc rankers, which inherit the guarantee on this model, order tokens better than the gate. Gate maps from a model trained on permuted labels are as close to the real maps as two real-label seeds are to each other. Under retraining, the tokens of the gate carry more task information than random tokens and, on shifted crops, more than central ones, but no more than the tokens chosen by hidden attention. We therefore read the gate as a certified support set produced by mask supervision. Splitting the Oracle Impurity Score into own-concept and cross-concept parts reveals differences the aggregate score misses. Partitioning the readout reduced leakage slightly under an earlier recipe, and the final model sits at, rather than below, the level implied by correlations among the true concepts.

## CaMBRAIN: Real-time, Continuous EEG Inference with Causal State Space Models

DevFeed: [CaMBRAIN: Real-time, Continuous EEG Inference with Causal State Space Models](<https://devfeed.tech/articles/cambrain-real-time-continuous-eeg-inference-with-causal-state-space-models-137258.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2605.28792>)

Author: Abhilash Durgam, Nyle Siddiqui, Jeffrey A. Chan-Santiago, Qiushi Fu, Elakkat D. Gireesh, Mubarak Shah

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Mamba](<https://devfeed.tech/topics/mamba.md>), [streaming data analytics](<https://devfeed.tech/topics/streaming-data-analytics.md>), [real-time](<https://devfeed.tech/topics/real-time.md>)

Tags: [continuous](<https://devfeed.tech/tags/continuous.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-hc](<https://devfeed.tech/tags/cs-hc.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [deep-learning](<https://devfeed.tech/tags/deep-learning.md>), [eeg](<https://devfeed.tech/tags/eeg.md>), [inference](<https://devfeed.tech/tags/inference.md>), [mamba](<https://devfeed.tech/tags/mamba.md>), [memory](<https://devfeed.tech/tags/memory.md>), [range](<https://devfeed.tech/tags/range.md>), [real-time](<https://devfeed.tech/tags/real-time.md>), [state-space-models](<https://devfeed.tech/tags/state-space-models.md>), [streaming](<https://devfeed.tech/tags/streaming.md>), [throughput](<https://devfeed.tech/tags/throughput.md>), [time-complexity](<https://devfeed.tech/tags/time-complexity.md>), [train](<https://devfeed.tech/tags/train.md>), [training](<https://devfeed.tech/tags/training.md>), [variable](<https://devfeed.tech/tags/variable.md>)

### AI overview

CaMBRAIN is a causal Mamba state space model designed to process variable-length EEG continuously in real time. It addresses the quadratic sequence scaling of attention-based models and the limited context of fixed-length sliding windows, using linear-time inference. Because brief EEG events may be separated by minutes, the authors introduce a multi-stage self-supervised training pipeline intended to teach the model's hidden state to retain long-range context; they argue reconstruction-focused objectives do not do this adequately for streaming models. The abstract reports state-of-the-art results on three EEG datasets and throughput over ten times higher than existing models. These claims are reported in the abstract; detailed methods and evaluation evidence are not included in the supplied excerpt.

### Source excerpt

arXiv:2605.28792v2 Announce Type: replace Abstract: Electroencephalography (EEG) is a critical, non-invasive method to monitor electrical brain activity. EEGs can span anywhere from a couple seconds to multiple hours, posing a major hurdle for existing deep learning methods due to two major factors: (1) existing EEG models are predominantly built upon the attention mechanism, incurring quadratic scaling as the sequence length increases, and (2) raw EEG signals must be processed in a sliding-window fashion due to fixed-length input requirements, preventing global understanding of the entire signal. To this extent, we propose CaMBRAIN - the first Causal, Mamba-based state space model (SSM) capable of real-time inference of EEG signals, arguing that bidirectional approaches are needlessly expensive given the causal, unidirectional nature of EEG. However, training such a model is non-trivial, as crucial EEG events can be extremely brief - within fractions of a second - yet separated by long intervals spanning minutes. Current EEG methods use self-supervised objectives that optimize for signal reconstruction, but these are not well suited for streaming SSMs; they fail to explicitly train the hidden state to retain the salient long-range context needed for streaming inference. We therefore introduce a multi-stage self-supervised training pipeline specifically tailored to encourage long-range memory retention and strong performance on EEG signals, while preserving the linear-time complexity of state space models. CaMBRAIN achieves state-of-the-art (SOTA) results across 3 different EEG datasets with >10x higher throughput than existing models, enabling the first model capable of long-range, continuous inference of variable-length EEG signals.

## Algebraic Interaction: A Recipe for Neural Architecture Generation

DevFeed: [Algebraic Interaction: A Recipe for Neural Architecture Generation](<https://devfeed.tech/articles/algebraic-interaction-a-recipe-for-neural-architecture-generation-137321.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.18649>)

Author: Haonan Dong, Angelica I Aviles-Rivero

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [model architecture](<https://devfeed.tech/topics/model-architecture.md>), [Neural Network](<https://devfeed.tech/topics/neural-network.md>), [Mamba](<https://devfeed.tech/topics/mamba.md>)

Tags: [algebra](<https://devfeed.tech/tags/algebra.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [attention](<https://devfeed.tech/tags/attention.md>), [composition](<https://devfeed.tech/tags/composition.md>), [convolution](<https://devfeed.tech/tags/convolution.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [design](<https://devfeed.tech/tags/design.md>), [efficiency](<https://devfeed.tech/tags/efficiency.md>), [faster](<https://devfeed.tech/tags/faster.md>), [feature](<https://devfeed.tech/tags/feature.md>), [flow](<https://devfeed.tech/tags/flow.md>), [foundation](<https://devfeed.tech/tags/foundation.md>), [framework](<https://devfeed.tech/tags/framework.md>), [inference](<https://devfeed.tech/tags/inference.md>), [information](<https://devfeed.tech/tags/information.md>), [interaction](<https://devfeed.tech/tags/interaction.md>), [linear](<https://devfeed.tech/tags/linear.md>), [mamba](<https://devfeed.tech/tags/mamba.md>), [models](<https://devfeed.tech/tags/models.md>), [neural](<https://devfeed.tech/tags/neural.md>), [recipe](<https://devfeed.tech/tags/recipe.md>)

### AI overview

The paper proposes an algebraic framework that separates positional routing from feature interactions in neural architectures. Its "Algebraic Recipe" keeps an existing model's routing and operator composition fixed while replacing its feature algebra, so information flow is preserved while computational and symmetry properties can change. In the reported applications, structure-constant and Clifford algebras improve linear-attention accuracy at comparable parameter counts; structure-constant algebras make Vision Mamba training and inference up to 3x faster at competitive accuracy; and an SO(3)-representation algebra builds rotational equivariance by construction. These claims are based on the supplied abstract.

### Source excerpt

arXiv:2611.18649v1 Announce Type: cross Abstract: We introduce an algebraic foundation for neural architecture design in which convolution, attention, state-space models, and equivariant layers are expressed through compositions of bilinear product operators over two structures: a position algebra governing positional routing and a feature algebra governing feature interactions. This decomposition makes routing, feature algebra, and operator composition explicit design variables.We focus on the feature-algebra axis through an Algebraic Recipe: fix the routing and operator composition of an existing architecture while replacing its feature algebra. This preserves the information-flow structure while allowing computational and symmetry properties to be modified through algebraic specification.We instantiate the recipe in linear attention, Mamba, and equivariant networks. Structure-constant and Clifford algebras improve accuracy at comparable parameter count in linear attention; structure-constant algebras yield up to $3\times$ faster training and inference in Vision Mamba at competitive accuracy; and an $\mathrm{SO}(3)$-representation algebra provides rotational equivariance by construction. These results demonstrate that feature algebra alone can control accuracy, efficiency, and symmetry without redesigning positional routing.This turns algebraic structure into an explicit design variable and reduces architecture design, within our framework, to algebraic specification.

## Integrated GPU Core for Mamba-3

DevFeed: [Integrated GPU Core for Mamba-3](<https://devfeed.tech/articles/integrated-gpu-core-for-mamba-3-139044.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.20409>)

Author: Shiva Raja, William Desueza, Ajay Joshi

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Mamba](<https://devfeed.tech/topics/mamba.md>), [GPU](<https://devfeed.tech/topics/gpu.md>), [Tensor Cores](<https://devfeed.tech/topics/tensor-cores.md>)

Tags: [core](<https://devfeed.tech/tags/core.md>), [cs-ar](<https://devfeed.tech/tags/cs-ar.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [data-movement](<https://devfeed.tech/tags/data-movement.md>), [decoding](<https://devfeed.tech/tags/decoding.md>), [energy](<https://devfeed.tech/tags/energy.md>), [execution](<https://devfeed.tech/tags/execution.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [hybrid](<https://devfeed.tech/tags/hybrid.md>), [inference](<https://devfeed.tech/tags/inference.md>), [kernel](<https://devfeed.tech/tags/kernel.md>), [long-context](<https://devfeed.tech/tags/long-context.md>), [mamba](<https://devfeed.tech/tags/mamba.md>), [memory-latency](<https://devfeed.tech/tags/memory-latency.md>), [mimo](<https://devfeed.tech/tags/mimo.md>), [model](<https://devfeed.tech/tags/model.md>), [pipelines](<https://devfeed.tech/tags/pipelines.md>), [power](<https://devfeed.tech/tags/power.md>), [power-modeling](<https://devfeed.tech/tags/power-modeling.md>), [prefill](<https://devfeed.tech/tags/prefill.md>), [recurrent](<https://devfeed.tech/tags/recurrent.md>), [rtl](<https://devfeed.tech/tags/rtl.md>)

### AI overview

RescuCore is a proposed GPU-integrated execution unit for Mamba-3 inference, targeting inefficiency from fragmented kernels, data movement, and register pressure. It shares SM sub-partition resources with CUDA and Tensor Cores: dense projections stay on conventional GPU resources, while RescuCore pipelines discretization, normalization, RoPE, MIMO computation, recurrent state updates, and output gating. The authors report that BF16 preserves model accuracy, with an estimated area overhead of 5.9% of an A100 die at state size 32 and MIMO rank 4. In simulation and synthesis-informed evaluation, RescuCore achieves 1.4-9.4x speedup and 1.4-5.0x energy reduction in prefill, and 1.3-2.8x speedup and 1.2-2.6x energy reduction in autoregressive decoding, compared with conventional GPU execution.

### Source excerpt

arXiv:2611.20409v1 Announce Type: cross Abstract: Mamba-3, the latest Mamba state-space model (SSM), enables efficient long-context modeling, matching Mamba-2 perplexity with half the state size and increasing decoding arithmetic intensity. However, its multi-input multi-output (MIMO) computation, exponential-trapezoidal discretization, and rotary positional embeddings (RoPE) introduce dense, element-wise, and sequential operations. Despite kernel fusion, conventional GPUs execute these operations inefficiently. Our characterization of Mamba-3 hybrid models reveals numerous kernel launches, substantial data movement, and increased register pressure that reduces warp occupancy and memory-latency hiding. We propose RescuCore, a specialized GPU execution unit for Mamba-3 inference that retains other hybrid model layers on existing GPU resources. Integrated alongside CUDA Cores and Tensor Cores within each Streaming Multiprocessor (SM) sub-partition, RescuCore shares the register file, shared memory, and warp schedulers. Dense projections remain on conventional GPU resources, while RescuCore pipelines discretization, normalization, RoPE, MIMO computation, recurrent state updates, and output gating to reduce kernel fragmentation and intermediate data movement. We select BF16 after evaluating multiple numerical formats, preserving model accuracy with an estimated area overhead of 5.9% of an A100 die for state size 32 and MIMO rank 4. We evaluate RescuCore using cycle-accurate Accel-Sim simulation, RTL synthesis, and synthesis-informed AccelWattch power modeling. Compared with conventional GPU execution, RescuCore achieves 1.4-9.4x speedup and 1.4-5.0x energy reduction during prefill, and 1.3-2.8x speedup and 1.2-2.6x energy reduction during autoregressive decoding. These results demonstrate the potential of GPU-integrated specialized execution for SSM-based hybrid models.

## A Hybrid State Space-Attention Architecture for Forecasting the Trajectories of Battery Degradation

DevFeed: [A Hybrid State Space-Attention Architecture for Forecasting the Trajectories of Battery Degradation](<https://devfeed.tech/articles/a-hybrid-state-space-attention-architecture-for-forecasting-the-trajectories-of-battery-degradation-137398.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.19116>)

Author: Shrey Patel, Bhargav Srinivas Rao, Samet Ayhan

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Mamba](<https://devfeed.tech/topics/mamba.md>), [reproducibility](<https://devfeed.tech/topics/reproducibility.md>), [Hyperparameter optimization](<https://devfeed.tech/topics/hyperparameter-optimization.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [aging](<https://devfeed.tech/tags/aging.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [attention](<https://devfeed.tech/tags/attention.md>), [battery](<https://devfeed.tech/tags/battery.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cycles](<https://devfeed.tech/tags/cycles.md>), [data](<https://devfeed.tech/tags/data.md>), [efficiency](<https://devfeed.tech/tags/efficiency.md>), [embeddings](<https://devfeed.tech/tags/embeddings.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [forecasting](<https://devfeed.tech/tags/forecasting.md>), [health](<https://devfeed.tech/tags/health.md>), [hybrid](<https://devfeed.tech/tags/hybrid.md>), [ion](<https://devfeed.tech/tags/ion.md>), [language](<https://devfeed.tech/tags/language.md>), [language-model](<https://devfeed.tech/tags/language-model.md>), [large](<https://devfeed.tech/tags/large.md>), [physics-chem-ph](<https://devfeed.tech/tags/physics-chem-ph.md>), [space](<https://devfeed.tech/tags/space.md>), [state](<https://devfeed.tech/tags/state.md>)

### AI overview

DecayFormer forecasts a battery cell's full state-of-health trajectory from early charge-discharge cycles using a compact hybrid architecture. It combines cycle-level selective recurrence--with one scalar state per channel--and attention queried by the recurrent state, alongside state-of-charge interval statistics. A master curve head jointly estimates lifetime and a monotonically non-increasing SOH profile over normalized lifetime. On BatteryLife's 22 aging-condition-exclusive folds, the study used one run per fold and one fixed configuration, with no fold-specific tuning. Mean test MAPE was 2.25% for Li-ion, 2.76% for CALB, 0.95% for Na-ion, and 7.19% for Zn-ion. The abstract presents these results as evidence of potential across chemistries and aging conditions; performance varies by chemistry, with the highest reported error on Zn-ion.

### Source excerpt

arXiv:2611.19116v1 Announce Type: new Abstract: Early-life battery degradation trajectory forecasting aims to predict a battery cell's full state-of-health (SOH) trajectory from its initial charge-discharge cycles, potentially reducing cell qualification and evaluation time from months to days. Despite recent advances, accurate forecasting remains challenging for emerging battery chemistries, data-scarce domains, and cells exhibiting degradation behaviors beyond those observed during training. Existing state-of-the-art approaches increasingly rely on model scaling, large language model-derived aging-condition embeddings, and extensive fold-specific hyperparameter tuning, raising concerns about computational efficiency, reproducibility, and generalization. This work introduces DecayFormer, a compact hybrid state-space and attention architecture designed for early-life degradation trajectory forecasting. DecayFormer combines cycle-level selective recurrence, using a single scalar state per channel, with an attention mechanism queried by the recurrent state. It further incorporates state-of-charge (SOC) interval statistics and a master curve head that jointly estimates a scalar lifetime and a monotonically non-increasing SOH profile parameterized by normalized lifetime. We evaluate DecayFormer on the BatteryLife benchmark using all 22 aging-condition-exclusive folds, with a single run per fold and one fixed configuration across all folds, without fold-specific tuning. DecayFormer achieves mean test MAPE values of 2.25% on Li-ion, 2.76% on CALB, 0.95% on Na-ion, and 7.19% on Zn-ion. These results demonstrate the potential of compact sequence models to deliver accurate early-life degradation forecasts across diverse battery chemistries and aging conditions, without relying on extensive model scaling or per-fold optimization.

## Mechanistic Interpretability of Mambas Selective Gate in Financial Markets: A Leakage-Controlled Study

DevFeed: [Mechanistic Interpretability of Mambas Selective Gate in Financial Markets: A Leakage-Controlled Study](<https://devfeed.tech/articles/mechanistic-interpretability-of-mambas-selective-gate-in-financial-markets-a-leakage-controlled-study-138748.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.18599>)

Author: Dip Roy, Rajiv Misra, Sanjay Kumar Singh, Anisha Roy

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Mamba](<https://devfeed.tech/topics/mamba.md>), [Finance](<https://devfeed.tech/topics/finance.md>)

Tags: [autoencoders](<https://devfeed.tech/tags/autoencoders.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [financial](<https://devfeed.tech/tags/financial.md>), [financial-markets](<https://devfeed.tech/tags/financial-markets.md>), [mamba](<https://devfeed.tech/tags/mamba.md>), [mechanistic-interpretability](<https://devfeed.tech/tags/mechanistic-interpretability.md>), [state-space-models](<https://devfeed.tech/tags/state-space-models.md>), [time-series](<https://devfeed.tech/tags/time-series.md>)

### AI overview

With leakage controls, SAE features from a six-layer Mamba recover the HMM market-regime label at F1 0.398, close to a raw-input logistic baseline (0.435) and an SAE trained on an untrained Mamba (0.364), rather than the uncontrolled F1 of 1.00. A 2x2 attribution analysis links the near-perfect result to the combination of auxiliary regime supervision and inputs overlapping the label definition; neither alone exceeds about 0.40. The residual selective-gate regime effect is statistically detectable but economically negligible and uniform across layers. The study also reports that an apparent polysemanticity gradient disappears under continuous measures, while a modest feature subset survives controls. It proposes a leakage-control protocol for interpretability evaluations of financial state-space models.

### Source excerpt

arXiv:2611.18599v1 Announce Type: new Abstract: Selectively gated state-space models (SSMs), notably Mamba, are increasingly used for financial time series, yet how they internally represent market regimes (bull, bear, sideways) remains opaque. This paper applies Mechanistic Interpretability (MI) to a six-layer Mamba trained on approximately twenty-one years of daily data for a point-in-time set of S&P 500 constituents plus macroeconomic and Fama-French factors, decomposing per-layer hidden states with Top-K Sparse Autoencoders (SAEs). Our central contribution is methodological and cautionary. A prior, uncontrolled configuration recovered market regimes from SAE features at F1 = 1.00. Under a controlled protocol that (i) removes the three input features that overlap the HMM label definition, (ii) disables the auxiliary regime-classification head, (iii) uses a point-in-time constituent universe, and (iv) evaluates against labels disjoint from the inputs, the same regime label is recovered at only F1 = 0.398, indistinguishable from a logistic baseline on raw inputs (0.435) and from an SAE trained on a randomly initialized, untrained Mamba (0.364). A 2x2 attribution grid shows the near-perfect result is an interaction of auxiliary supervision and label-defining inputs: neither factor alone exceeds F1 ~ 0.40. Effect sizes with block-bootstrap inference show the residual selective-gate (Delta t) regime effect is statistically detectable but economically negligible and uniform across layers rather than layer-specialized; the apparent polysemanticity gradient disappears under continuous, theory-grounded measures. We contribute a reproducible leakage-control protocol for interpretability evaluation of financial SSMs, a quantified demonstration that previously reported regime structure is a label-contamination artifact, and a modest economically grounded feature subset that genuinely survives the controls.

## RiverScan: Learning to Accumulate on River Networks

DevFeed: [RiverScan: Learning to Accumulate on River Networks](<https://devfeed.tech/articles/riverscan-learning-to-accumulate-on-river-networks-135855.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.18005>)

Author: He Zhu, Zhiyi Gao, Youhuan Yang, Han Quan, Ming Li

Published: 2026-11-16T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Predictive AI](<https://devfeed.tech/topics/predictive-ai.md>), [model architecture](<https://devfeed.tech/topics/model-architecture.md>), [Representation learning](<https://devfeed.tech/topics/representation-learning.md>), [Mamba](<https://devfeed.tech/topics/mamba.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [area](<https://devfeed.tech/tags/area.md>), [bias](<https://devfeed.tech/tags/bias.md>), [challenge](<https://devfeed.tech/tags/challenge.md>), [combines](<https://devfeed.tech/tags/combines.md>), [connectivity](<https://devfeed.tech/tags/connectivity.md>), [contributing](<https://devfeed.tech/tags/contributing.md>), [contributions](<https://devfeed.tech/tags/contributions.md>), [controls](<https://devfeed.tech/tags/controls.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [distributed](<https://devfeed.tech/tags/distributed.md>), [effective](<https://devfeed.tech/tags/effective.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [graph](<https://devfeed.tech/tags/graph.md>), [hierarchy](<https://devfeed.tech/tags/hierarchy.md>), [inductive](<https://devfeed.tech/tags/inductive.md>), [information](<https://devfeed.tech/tags/information.md>), [large](<https://devfeed.tech/tags/large.md>), [learning](<https://devfeed.tech/tags/learning.md>), [local](<https://devfeed.tech/tags/local.md>), [networks](<https://devfeed.tech/tags/networks.md>)

### AI overview

RiverScan models runoff by accumulating subcatchment information along river networks. Its spatial operator uses area-weighted selective state-space recurrence: it combines local temporal representations with upstream states in topological order, weighting contributions by contributing-area fractions and learning how much upstream information to retain. In experiments across 608 Danube basin stations, it outperformed the evaluated temporal and graph baselines overall. Structural controls and ablations indicate that accumulation aligned with hydrological structure improves prediction, and that correct topology becomes more beneficial higher in the river-network hierarchy. The supplied abstract does not specify the baselines or quantitative performance differences.

### Source excerpt

arXiv:2611.18005v1 Announce Type: new Abstract: Accurate runoff prediction across large river basins requires models to capture heterogeneous local hydrological responses and their accumulation downstream. River-network connectivity provides a natural structure for this task, but using it effectively requires learning how to combine contributions from subcatchments of different sizes. We frame this challenge as learning to accumulate and introduce RiverScan, a spatial operator based on area-weighted selective state-space recurrence. RiverScan recursively combines local temporal representations with accumulated upstream states in topological order. Contributing-area fractions account for subcatchment size, while learned retention controls how upstream information persists downstream. Experiments on 608 stations in the Danube basin show that RiverScan outperforms the evaluated temporal and graph baselines in overall predictive performance. Structural controls and ablations demonstrate that composing heterogeneous subcatchment representations in a manner aligned with hydrological accumulation substantially improves runoff prediction, with the benefits of correct topology growing along the river-network hierarchy. These findings establish physically structured accumulation as an effective inductive bias for hydrological modeling and motivate finer-grained, fully distributed AI-based hydrological models.

## Climate Temporal Temperature Prediction via an Adaptive Stacked Bidirectional Residual Mamba Network

DevFeed: [Climate Temporal Temperature Prediction via an Adaptive Stacked Bidirectional Residual Mamba Network](<https://devfeed.tech/articles/climate-temporal-temperature-prediction-via-an-adaptive-stacked-bidirectional-residual-mamba-network-133110.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.16322>)

Author: Yongxiang Lei

Published: 2026-11-13T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Mamba](<https://devfeed.tech/topics/mamba.md>), [Climate / Weather / Ocean Modeling](<https://devfeed.tech/topics/climate-weather-ocean-modeling.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Predictive AI](<https://devfeed.tech/topics/predictive-ai.md>)

Tags: [adaptive](<https://devfeed.tech/tags/adaptive.md>), [algorithm](<https://devfeed.tech/tags/algorithm.md>), [approximation](<https://devfeed.tech/tags/approximation.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [attention](<https://devfeed.tech/tags/attention.md>), [climate](<https://devfeed.tech/tags/climate.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [deep-learning](<https://devfeed.tech/tags/deep-learning.md>), [forecasting](<https://devfeed.tech/tags/forecasting.md>), [framework](<https://devfeed.tech/tags/framework.md>), [hardware](<https://devfeed.tech/tags/hardware.md>), [implement](<https://devfeed.tech/tags/implement.md>), [inference](<https://devfeed.tech/tags/inference.md>), [mamba](<https://devfeed.tech/tags/mamba.md>), [mlp](<https://devfeed.tech/tags/mlp.md>), [models](<https://devfeed.tech/tags/models.md>), [network](<https://devfeed.tech/tags/network.md>), [parallel](<https://devfeed.tech/tags/parallel.md>), [prediction](<https://devfeed.tech/tags/prediction.md>), [temperature](<https://devfeed.tech/tags/temperature.md>), [temporal](<https://devfeed.tech/tags/temporal.md>)

### AI overview

Deep DualMamba is a temperature-forecasting model that uses adaptive bidirectional Mamba encoders and a hardware-aware parallel recurrent algorithm. The authors say its selective state-space design avoids attention and MLP layers, has linear complexity with sequence length, and improves inference throughput. They describe theoretical analysis of approximation and convergence, and a forecasting pipeline built with historical data from UK monitoring stations. The abstract provides no quantitative results or detailed comparison with other models.

### Source excerpt

arXiv:2611.16322v1 Announce Type: new Abstract: Accurate temperatures forecasting relies on traditional meteorological parameters that are essential for monitoring weather informatics and guiding forecasting efforts. This study presents Deep DualMamba, a novel deep learning framework for temporal temperature prediction, built upon an enhanced variant of the recently proposed Mamba network. Unlike conventional Transformer-based architectures, which dominate foundation models but suffer from computational inefficiencies on long sequences, Deep DualMamba leverages selective structured state space models (SSMs) to achieve both scalability and performance. We identify a core limitation of prior subquadratic-time models-namely, their inability to perform long-term and robust forecasting and introduce two key innovations. First, we propose an adaptive bidirectional Mamba as encoders to learn complex and efficient features. Second, we develop a hardware-aware, parallel recurrent algorithm to compensate for the loss of convolutional efficiency, ensuring practical viability at scale. Last, the Deep DualMamba's approximation ability and convergence is theoretical analyzed. These design choices culminate in a simplified, attention- and MLP-free neural architecture that exhibits linear complexity with sequence length and significantly improved inference throughput. To demonstrate its applicability in climate informatics, we implement a full forecasting pipeline using historical meteorological data from UK monitoring stations.

## Symmetry in Time: A Representation Theory of Linear Recurrent Memory

DevFeed: [Symmetry in Time: A Representation Theory of Linear Recurrent Memory](<https://devfeed.tech/articles/symmetry-in-time-a-representation-theory-of-linear-recurrent-memory-133185.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.17162>)

Author: Anif N. Shikder, Ramit Dey, Jan Minac, Lyle E. Muller

Published: 2026-11-13T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Mamba](<https://devfeed.tech/topics/mamba.md>), [Transformers](<https://devfeed.tech/topics/transformers.md>)

Tags: [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [blocks](<https://devfeed.tech/tags/blocks.md>), [complex](<https://devfeed.tech/tags/complex.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-ne](<https://devfeed.tech/tags/cs-ne.md>), [generators](<https://devfeed.tech/tags/generators.md>), [groups](<https://devfeed.tech/tags/groups.md>), [inference](<https://devfeed.tech/tags/inference.md>), [input](<https://devfeed.tech/tags/input.md>), [linear](<https://devfeed.tech/tags/linear.md>), [math-rt](<https://devfeed.tech/tags/math-rt.md>), [matrices](<https://devfeed.tech/tags/matrices.md>), [memory](<https://devfeed.tech/tags/memory.md>), [models](<https://devfeed.tech/tags/models.md>), [order](<https://devfeed.tech/tags/order.md>), [parallel](<https://devfeed.tech/tags/parallel.md>), [protein](<https://devfeed.tech/tags/protein.md>), [qubit](<https://devfeed.tech/tags/qubit.md>), [recurrent](<https://devfeed.tech/tags/recurrent.md>), [state](<https://devfeed.tech/tags/state.md>), [symmetry](<https://devfeed.tech/tags/symmetry.md>), [theory](<https://devfeed.tech/tags/theory.md>), [time](<https://devfeed.tech/tags/time.md>)

### AI overview

The paper argues that diagonal real- or complex-valued state space models (SSMs) cannot track noncommutative groups because their transitions commute and lose input order. It formalizes group tracking through representation theory: an SSM tracks a group at every sequence length exactly when that group is a quotient of the group generated by its transition matrices. Such transitions must split into independent real, complex, and quaternion blocks. The authors introduce HyperSSM, adding quaternion channels while retaining linear-time inference and parallel training. They report exact results on recent group-tracking benchmarks with at most five state numbers, including S₅ with 32 generators, where previous SSMs perform at chance, and generalization to sequences 25 times longer than training. They also report interpreting learned transitions to recover gyroscope calibration, imperfect qubit-gate rotations, and protein-backbone geometry.

### Source excerpt

arXiv:2611.17162v1 Announce Type: new Abstract: State space models (SSMs) are an emerging alternative to Transformers. SSMs offer linear-time inference and parallel training, but most implementations make the parallel scan efficient by making the recurrence diagonal: each channel of the hidden state is multiplied by an input-dependent real or complex number. These numbers commute, so the state records which inputs occurred but not their order. We formalize this with representation theory, which explains and fixes the state-tracking failures previously reported for SSMs. The state update defines a linear representation of the monoid of input sequences, and we show that an SSM tracks a group at every sequence length exactly when that group is a quotient of the group generated by its transition matrices. Two consequences follow. Diagonal real- and complex-valued SSMs can track only commutative groups, while the transition matrices of any SSM that can track a group must split into independent blocks over the real, complex, and quaternion numbers. We then introduce HyperSSM, which adds quaternion channels while keeping linear-time inference and parallel training. Trained only on final answers, HyperSSM solves the group tasks of recent state-tracking benchmarks exactly with at most five state numbers. This includes $S_5$ with 32 generators, where previous SSMs are at chance, and HyperSSM generalizes to sequences 25 times longer than those seen in training. Because the theory says what the transitions must encode, the learned weights of the linear recurrence can be interpreted directly. We demonstrate this by recovering the calibration of a real gyroscope, the rotations of imperfect qubit gates and the geometry of protein backbones.

## Time Flies When You Parallelise: Pareto-efficient Time Series Foundation Models with Linear Recurrences

DevFeed: [Time Flies When You Parallelise: Pareto-efficient Time Series Foundation Models with Linear Recurrences](<https://devfeed.tech/articles/time-flies-when-you-parallelise-pareto-efficient-time-series-foundation-models-with-linear-recurrences-133161.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.16934>)

Author: Torben Berndt, Lingyi Yang, Federico Nardi, Yi Heng Lim, AJ Shears, Benjamin Walker

Published: 2026-11-13T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Time Series](<https://devfeed.tech/topics/time-series.md>), [Mamba](<https://devfeed.tech/topics/mamba.md>)

Tags: [absolute](<https://devfeed.tech/tags/absolute.md>), [attention](<https://devfeed.tech/tags/attention.md>), [bench](<https://devfeed.tech/tags/bench.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [forecasting](<https://devfeed.tech/tags/forecasting.md>), [frontier](<https://devfeed.tech/tags/frontier.md>), [inference](<https://devfeed.tech/tags/inference.md>), [inference-costs](<https://devfeed.tech/tags/inference-costs.md>), [linear](<https://devfeed.tech/tags/linear.md>), [mamba](<https://devfeed.tech/tags/mamba.md>), [models](<https://devfeed.tech/tags/models.md>), [parallel](<https://devfeed.tech/tags/parallel.md>), [performance](<https://devfeed.tech/tags/performance.md>), [probabilistic-forecasting](<https://devfeed.tech/tags/probabilistic-forecasting.md>), [quantile](<https://devfeed.tech/tags/quantile.md>), [recurrent](<https://devfeed.tech/tags/recurrent.md>), [runtime](<https://devfeed.tech/tags/runtime.md>), [time-series](<https://devfeed.tech/tags/time-series.md>), [time-series-foundation-models](<https://devfeed.tech/tags/time-series-foundation-models.md>), [training](<https://devfeed.tech/tags/training.md>), [zero-shot](<https://devfeed.tech/tags/zero-shot.md>)

### AI overview

The ci-proto family uses Mamba-3 linear state recurrences to enable chunk-wise parallel computation over time during training, with inference costs that scale linearly with sequence length. Its architecture alternates recurrent temporal processing with cross-variate attention for multivariate probabilistic forecasting with past-only and future-known covariates. On fev-bench's 100 forecasting tasks, ci-proto XL reports the best performance among time series foundation models; all five variants form the reported Pareto frontier for mean scaled quantile loss and mean absolute scaled error against both parameter count and median runtime. These findings are from the supplied abstract.

### Source excerpt

arXiv:2611.16934v1 Announce Type: new Abstract: Time series foundation models have demonstrated strong zero-shot forecasting performance, but computational efficiency remains a central architectural challenge. Temporal self-attention incurs quadratic cost in the number of temporal tokens, while nonlinear recurrent state transitions restrict parallel computation along the temporal dimension. We introduce ci-proto, a family of time series foundation models based on Mamba-3, whose linear state recurrences support chunk-wise parallel-in-time computation during training, and inference costs that scale linearly with sequence length. The architecture alternates recurrent temporal processing with cross-variate attention, supporting multivariate probabilistic forecasting with past-only and future-known covariates. Across fev-bench's 100 forecasting tasks, ci-proto XL sets a new state of the art among time series foundation models. The five ci-proto variants also make up the entire Pareto frontier for mean scaled quantile loss and mean absolute scaled error against both parameter count and median runtime.

## MS-rPPG: Multi-spectral State Space Model for Remote Photoplethysmography in Driver Monitoring Systems

DevFeed: [MS-rPPG: Multi-spectral State Space Model for Remote Photoplethysmography in Driver Monitoring Systems](<https://devfeed.tech/articles/ms-rppg-multi-spectral-state-space-model-for-remote-photoplethysmography-in-driver-monitoring-systems-132810.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2606.21115>)

Author: Jiho Choi, Sang Jun Lee

Published: 2026-11-13T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Mamba](<https://devfeed.tech/topics/mamba.md>)

Tags: [camera](<https://devfeed.tech/tags/camera.md>), [car](<https://devfeed.tech/tags/car.md>), [channel](<https://devfeed.tech/tags/channel.md>), [cross](<https://devfeed.tech/tags/cross.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [dataset](<https://devfeed.tech/tags/dataset.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [driver](<https://devfeed.tech/tags/driver.md>), [driving](<https://devfeed.tech/tags/driving.md>), [eess-iv](<https://devfeed.tech/tags/eess-iv.md>), [environments](<https://devfeed.tech/tags/environments.md>), [framework](<https://devfeed.tech/tags/framework.md>), [health](<https://devfeed.tech/tags/health.md>), [health-monitoring](<https://devfeed.tech/tags/health-monitoring.md>), [infrared](<https://devfeed.tech/tags/infrared.md>), [linear](<https://devfeed.tech/tags/linear.md>), [mamba](<https://devfeed.tech/tags/mamba.md>), [measuring](<https://devfeed.tech/tags/measuring.md>), [model](<https://devfeed.tech/tags/model.md>), [monitoring](<https://devfeed.tech/tags/monitoring.md>), [remote](<https://devfeed.tech/tags/remote.md>), [state](<https://devfeed.tech/tags/state.md>)

### AI overview

MS-rPPG combines RGB and near-infrared face video to estimate heart rate under variable lighting and frequent head movement in driving settings. Its cross-spectral linear modulation strategy uses frequency-domain analysis to combine features from both video types, while MS-Mamba models long-range temporal dependencies and cross-channel interactions. The authors collected MS-Drive from 50 participants driving and evaluated the method on MS-Drive and MR-NIRP Car. They report better robustness and heart-rate estimation accuracy than previous methods; the abstract provides no specific metrics.

### Source excerpt

arXiv:2606.21115v1 Announce Type: cross Abstract: Remote photoplethysmography (rPPG) is a camera-based technique for measuring physiological signals, particularly cardiac activity. From the remotely measured signals, heart rate can be estimated, which is crucial for health monitoring. In this study, we investigate a driver health monitoring system based on remote heart rate estimation. However, driving environments represent uncontrolled settings where videos are subject to varying illumination conditions and frequent head movements. We introduce MS-rPPG, a multi-spectral framework that combines RGB with near-infrared (NIR) face video to alleviate rPPG estimation under challenging driving conditions. To combine the complementary features from two spectral videos, we propose a cross-spectral linear modulation (CSLM) strategy based on frequency-domain analysis. Moreover, we introduce MS-Mamba, a novel state space model designed to effectively model long-range temporal dependencies while jointly capturing cross-channel interactions between multi-spectral features. We collected a real-world dataset called MS-Drive, which was recorded from 50 participants while driving the vehicle. The proposed method was evaluated on the MR-NIRP Car dataset and MS-Drive datasets. The experimental results indicate that MS-rPPG shows better robustness and heart rate estimation accuracy than previous methods, highlighting its promise for driver health monitoring. The codes are available at github.com/ziiho08/MS-rPPG.

## SinkState: On-the-Fly SVD of Recurrent States for Efficient Hybrid LLM Serving

DevFeed: [SinkState: On-the-Fly SVD of Recurrent States for Efficient Hybrid LLM Serving](<https://devfeed.tech/articles/sinkstate-on-the-fly-svd-of-recurrent-states-for-efficient-hybrid-llm-serving-128814.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.14246>)

Author: Haolin Jiang, Weihang Li, Zhongzhu Zhou, Jinwen Pan, Santosh Pandey, Hang Liu

Published: 2026-11-11T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>), [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Mamba](<https://devfeed.tech/topics/mamba.md>)

Tags: [adaptive](<https://devfeed.tech/tags/adaptive.md>), [attention](<https://devfeed.tech/tags/attention.md>), [attention-sink](<https://devfeed.tech/tags/attention-sink.md>), [compression](<https://devfeed.tech/tags/compression.md>), [cpu](<https://devfeed.tech/tags/cpu.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [decoding](<https://devfeed.tech/tags/decoding.md>), [efficiency](<https://devfeed.tech/tags/efficiency.md>), [hybrid](<https://devfeed.tech/tags/hybrid.md>), [keys](<https://devfeed.tech/tags/keys.md>), [linear](<https://devfeed.tech/tags/linear.md>), [llm](<https://devfeed.tech/tags/llm.md>), [llm-serving](<https://devfeed.tech/tags/llm-serving.md>), [mamba-2](<https://devfeed.tech/tags/mamba-2.md>), [memory](<https://devfeed.tech/tags/memory.md>), [model](<https://devfeed.tech/tags/model.md>), [models](<https://devfeed.tech/tags/models.md>), [queries](<https://devfeed.tech/tags/queries.md>), [rank](<https://devfeed.tech/tags/rank.md>), [recurrent](<https://devfeed.tech/tags/recurrent.md>), [serving](<https://devfeed.tech/tags/serving.md>)

### AI overview

SinkState exploits the finding that linear attention queries, keys, and values cluster in a stable, low-dimensional subspace, making recurrent states effectively low-rank with rank varying by state. It stores states adaptively in low-rank form during inference, uses buffered updates and a local SVD over the top subspace, and consolidates remaining components asynchronously on the CPU. Across four hybrid models with Mamba-2 and Gated DeltaNet layers, spanning 4B to 120B parameters, it reportedly cuts peak recurrent-state memory by up to 2.6x (1.9x on average) and speeds decoding by up to 1.58x (1.25x on average) versus Mamba-SSM and FLA, while maintaining accuracy.

### Source excerpt

arXiv:2611.14246v1 Announce Type: new Abstract: Hybrid models combine softmax attention with linear attention, which compresses history into fixed-size recurrent states to balance model quality and inference efficiency. However, linear attention has not pushed this compression to its limit: we find that the queries, keys, and values of linear layers concentrate in a stable, low-dimensional subspace, a phenomenon we call the linear attention sink, so each state is effectively low-rank, with a rank that varies across states. Yet existing serving systems keep these redundant states dense, reading and rewriting them in full at every decoding step, so their memory traffic grows with the batch size and dominates the decoding cost. Thus SinkState stores each recurrent state in an adaptive low-rank form throughout inference with buffered updates and a local SVD restricted to the top subspace. SinkState further consolidates the remaining components asynchronously on the CPU. On four hybrid models with Mamba-2 and Gated DeltaNet (GDN) layers, from 4B to 120B parameters, SinkState reduces the peak memory of recurrent states by up to 2.6x (1.9x on average) and speeds up decoding by up to 1.58x (1.25x on average) over Mamba-SSM and FLA, while maintaining accuracy.

## Seeing the Small: Deformable Duality Mamba for Retinal Lesion Segmentation

DevFeed: [Seeing the Small: Deformable Duality Mamba for Retinal Lesion Segmentation](<https://devfeed.tech/articles/seeing-the-small-deformable-duality-mamba-for-retinal-lesion-segmentation-128721.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.13514>)

Author: Chang Yu, Siqing Li, Yuanchen Liu, Zhenyu Bu, Luming Yang, Alper Yilmaz

Published: 2026-11-11T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Mamba](<https://devfeed.tech/topics/mamba.md>), [Image processing](<https://devfeed.tech/topics/image-processing.md>)

Tags: [adaptive](<https://devfeed.tech/tags/adaptive.md>), [context](<https://devfeed.tech/tags/context.md>), [cross-validation](<https://devfeed.tech/tags/cross-validation.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [dataset](<https://devfeed.tech/tags/dataset.md>), [ddr](<https://devfeed.tech/tags/ddr.md>), [dr](<https://devfeed.tech/tags/dr.md>), [duality](<https://devfeed.tech/tags/duality.md>), [easily](<https://devfeed.tech/tags/easily.md>), [global](<https://devfeed.tech/tags/global.md>), [hypothesis](<https://devfeed.tech/tags/hypothesis.md>), [irregular](<https://devfeed.tech/tags/irregular.md>), [mamba](<https://devfeed.tech/tags/mamba.md>), [modeling](<https://devfeed.tech/tags/modeling.md>), [organization](<https://devfeed.tech/tags/organization.md>), [performance](<https://devfeed.tech/tags/performance.md>), [pixels](<https://devfeed.tech/tags/pixels.md>), [points](<https://devfeed.tech/tags/points.md>), [protocol](<https://devfeed.tech/tags/protocol.md>), [regions](<https://devfeed.tech/tags/regions.md>), [small](<https://devfeed.tech/tags/small.md>)

### AI overview

D²Mamba targets tiny, irregular diabetic retinopathy lesions by combining non-causal state space duality with deformable Mamba scanning to capture global context and adapt scans to lesion shape. In five-fold cross-validation on the DDR dataset, using the same training protocol as representative segmentation baselines, it achieved 33.87% mIoU--2.99 percentage points above the strongest baseline. The largest reported gain was for microaneurysms: Dice improved by 10.06 percentage points. The abstract also reports component ablations; these results are limited to the stated dataset and evaluation.

### Source excerpt

arXiv:2611.13514v1 Announce Type: cross Abstract: Diabetic retinopathy (DR) lesions vary widely in shape, and microaneurysms (MAs) span only a few pixels. We hypothesize that modeling the spatial organization of lesion regions, with global context and shape-adaptive scanning, improves the segmentation of such small and irregular lesions. To test this, we propose D$^2$Mamba, which couples non-causal state space duality with deformable Mamba scanning. We evaluate it with five-fold cross-validation on the DDR dataset, comparing against representative segmentation baselines under an identical training protocol, and conduct ablations on each component. D$^2$Mamba achieved the best overall performance, with 33.87% mIoU, 2.99 percentage points above the strongest baseline. The largest gain appeared on MAs, the smallest and most easily missed lesions, where Dice improved by 10.06 percentage points. These results confirm our hypothesis that spatially adaptive state space modeling benefits tiny-lesion segmentation. Code and pretrained weights are available at https://github.com/PUREPANEL1/D2Mamba.

## Parallel Broyden Methods for Efficiently Evaluating Nonlinear State Space Models

DevFeed: [Parallel Broyden Methods for Efficiently Evaluating Nonlinear State Space Models](<https://devfeed.tech/articles/parallel-broyden-methods-for-efficiently-evaluating-nonlinear-state-space-models-128722.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.13523>)

Author: Ian Christopher Tanoh, Scott W. Linderman

Published: 2026-11-11T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Algorithms](<https://devfeed.tech/topics/algorithms.md>), [Mamba](<https://devfeed.tech/topics/mamba.md>), [Compression](<https://devfeed.tech/topics/compression.md>)

Tags: [algorithms](<https://devfeed.tech/tags/algorithms.md>), [approximation](<https://devfeed.tech/tags/approximation.md>), [compression](<https://devfeed.tech/tags/compression.md>), [convergence](<https://devfeed.tech/tags/convergence.md>), [coupling](<https://devfeed.tech/tags/coupling.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-na](<https://devfeed.tech/tags/cs-na.md>), [denoising](<https://devfeed.tech/tags/denoising.md>), [diffusion](<https://devfeed.tech/tags/diffusion.md>), [diffusion-models](<https://devfeed.tech/tags/diffusion-models.md>), [dimensions](<https://devfeed.tech/tags/dimensions.md>), [error](<https://devfeed.tech/tags/error.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [hardware](<https://devfeed.tech/tags/hardware.md>), [math-na](<https://devfeed.tech/tags/math-na.md>), [mcmc](<https://devfeed.tech/tags/mcmc.md>), [newton](<https://devfeed.tech/tags/newton.md>), [novel](<https://devfeed.tech/tags/novel.md>), [parallel](<https://devfeed.tech/tags/parallel.md>), [recovery](<https://devfeed.tech/tags/recovery.md>), [root](<https://devfeed.tech/tags/root.md>), [scaling](<https://devfeed.tech/tags/scaling.md>), [state-space-models](<https://devfeed.tech/tags/state-space-models.md>)

### AI overview

The proposed parallel Broyden method evaluates nonlinear state space models by solving a root-finding problem while approximating Jacobian blocks with diagonal-plus-low-rank terms. It builds these approximations from trajectory secants, avoiding automatic differentiation, and uses rank compression to support parallel scan. The authors prove finite-step recovery of the exact trajectory under compressed scan; their convergence analysis depends on compression and trajectory-aligned block approximation errors. In reported experiments on low-rank RNNs, parallel MCMC, and denoising diffusion models, the method takes fewer iterations than other quasi-Newton methods and is fastest among parallel methods when Jacobian evaluation is expensive or state dimensions are large. Its advantages are reported for approximately low-rank interactions or costly Jacobian evaluation.

### Source excerpt

arXiv:2611.13523v1 Announce Type: new Abstract: Nonlinear state space models are typically evaluated sequentially, limiting their ability to exploit modern parallel hardware. Recent work has shown that state space models can be evaluated in parallel by reformulating evaluation as a root-finding problem, which can be solved with a parallel form of Newton's method. However, exact Newton methods require expensive multiplication of dense Jacobian matrices, and quasi-Newton methods based on diagonal approximations, while more scalable, are often slow to converge because they neglect interactions across state dimensions. We introduce a novel quasi-Newton approach based on Broyden's method, which captures coupling terms with diagonal-plus-low-rank approximations to blocks of the Jacobian. We construct the approximations from trajectory secants, avoiding automatic differentiation while retaining the favorable scaling of diagonal approximations and remaining compatible with parallel scan through rank compression. We prove finite-step recovery of the exact trajectory under the compressed scan, and show that convergence is tracked by the compression error and by the block approximation error along the current trajectory error, rather than in spectral norm. Empirically, we show that our method is advantageous when interactions are approximately low rank or when Jacobian evaluation is expensive: on low-rank RNNs, parallel MCMC algorithms, and denoising diffusion models, it converges in fewer iterations than other quasi-Newton methods and is the fastest parallel method when Jacobian evaluation is expensive or state dimensions are large.

## MambaLoc: Lightweight Indoor Localization Using Cross-Modal Knowledge Distillation

DevFeed: [MambaLoc: Lightweight Indoor Localization Using Cross-Modal Knowledge Distillation](<https://devfeed.tech/articles/mambaloc-lightweight-indoor-localization-using-cross-modal-knowledge-distillation-125923.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.10360>)

Author: Mohab Bahnassy, Omar Saqr, Hamada Rizk, Moustafa Youssef

Published: 2026-11-10T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Networking](<https://devfeed.tech/sources/arxiv-networking.md>)

Topics: [Mamba](<https://devfeed.tech/topics/mamba.md>), [Post-training optimization](<https://devfeed.tech/topics/post-training-optimization.md>), [Physical AI](<https://devfeed.tech/topics/physical-ai.md>), [Local AI](<https://devfeed.tech/topics/local-ai.md>)

Tags: [alignment](<https://devfeed.tech/tags/alignment.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-ni](<https://devfeed.tech/tags/cs-ni.md>), [csi](<https://devfeed.tech/tags/csi.md>), [distillation](<https://devfeed.tech/tags/distillation.md>), [edge](<https://devfeed.tech/tags/edge.md>), [efficiency](<https://devfeed.tech/tags/efficiency.md>), [error](<https://devfeed.tech/tags/error.md>), [evaluations](<https://devfeed.tech/tags/evaluations.md>), [gaussian](<https://devfeed.tech/tags/gaussian.md>), [hardware](<https://devfeed.tech/tags/hardware.md>), [inference](<https://devfeed.tech/tags/inference.md>), [information](<https://devfeed.tech/tags/information.md>), [infrastructure](<https://devfeed.tech/tags/infrastructure.md>), [knowledge-distillation](<https://devfeed.tech/tags/knowledge-distillation.md>), [latency](<https://devfeed.tech/tags/latency.md>), [lightweight](<https://devfeed.tech/tags/lightweight.md>), [linear](<https://devfeed.tech/tags/linear.md>), [localization](<https://devfeed.tech/tags/localization.md>), [mamba](<https://devfeed.tech/tags/mamba.md>), [model](<https://devfeed.tech/tags/model.md>), [probabilistic](<https://devfeed.tech/tags/probabilistic.md>)

### AI overview

MambaLoc distills geometric knowledge from a UWB-Transformer teacher into a lightweight Wi-Fi CSI Mamba model, so UWB hardware is not needed at inference. A Feature Alignment Module bridges the RF modalities, while a Gaussian Mixture regression head represents multiple plausible locations caused by multipath. On a real-world dataset, it reports 0.230 m median localization error, 22.2% lower error than standalone Mamba, and 65.1% fewer FLOPs and 58% lower inference latency than Transformer-based approaches. The supplied text is an abstract, so it does not provide further evaluation details.

### Source excerpt

arXiv:2611.10360v1 Announce Type: new Abstract: Accurate indoor localization on resource-constrained edge devices remains challenging due to the tension between modeling fidelity and computational efficiency. Wi-Fi Channel State Information (CSI) enables low-cost deployment using existing Wi-Fi infrastructure but is heavily distorted by multipath propagation. Transformer-based solutions mitigate these effects but their quadratic complexity limits real-time edge deployment. Ultra-Wideband (UWB) provides geometrically precise ranging yet requires dedicated infrastructure. We present MambaLoc, a cross-modal distillation framework that improves the accuracy-efficiency balance in CSI-based localization. MambaLoc uses a linear-time Selective State Space Model (Mamba) as a lightweight CSI student, guided during training by a UWB-Transformer teacher. A Feature Alignment Module bridges the representational gap between heterogeneous RF modalities, allowing the CSI model to inherit UWB-derived geometric structure without UWB hardware at inference. A probabilistic Gaussian Mixture regression head models multi-modal spatial ambiguity from multipath effects, avoiding the physically implausible averaging of deterministic regression. Evaluations on a real-world dataset show MambaLoc achieves a median localization error of 0.230 m, reducing error by 22.2% over a standalone Mamba baseline, while requiring 65.1% fewer FLOPs and 58% lower inference latency than Transformer-based approaches.

[Next page](<https://devfeed.tech/topics/mamba.md?cursor=WyIyMDI2LTExLTEwVDA1OjAwOjAwKzAwOjAwIiwgImM3YjJkOTFmLWM1ODgtNDg0Yi1iMzVkLThhMjUzNDRmZTU5NSJd>)