# Representation learning

Representation learning is the process of learning compact, structured data representations that preserve essential information and make downstream tasks easier.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## RePAIR: Predictive Self-Supervised Representation Learning in Chess

DevFeed: [RePAIR: Predictive Self-Supervised Representation Learning in Chess](<https://devfeed.tech/articles/repair-predictive-self-supervised-representation-learning-in-chess-144905.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2606.11860>)

Author: Christoph Koller, Johannes F\\"urnkranz, Timo Bertram

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Representation learning](<https://devfeed.tech/topics/representation-learning.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Self-supervised learning](<https://devfeed.tech/topics/self-supervised-learning.md>), [Embeddings](<https://devfeed.tech/topics/embeddings.md>), [bert](<https://devfeed.tech/topics/bert.md>)

Tags: [autoencoders](<https://devfeed.tech/tags/autoencoders.md>), [bert](<https://devfeed.tech/tags/bert.md>), [chess](<https://devfeed.tech/tags/chess.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [learning](<https://devfeed.tech/tags/learning.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [representation-learning](<https://devfeed.tech/tags/representation-learning.md>), [sequence](<https://devfeed.tech/tags/sequence.md>), [space](<https://devfeed.tech/tags/space.md>)

### AI overview

RePAIR learns compact representations of consecutive chess positions by masking large portions of latent-state sequences, then using a lightweight predictor to repair the gaps in a lower-dimensional embedding space. The authors report that its encoder clusters meaningful chess concepts, and that reconstructing masked board states captures piece movements without reinforcement learning. They also use trajectories through the representation space to inspect games. These findings are reported for chess; the supplied excerpt does not establish performance on other sequential data.

### Source excerpt

arXiv:2606.11860v2 Announce Type: replace Abstract: In this paper, we introduce Representation Prediction via Autoencoding using Iterative Refinement (RePAIR) - a novel self-supervised representation learning architecture that synthesizes Masked Autoencoders (MAE), Joint Embedding Predictive Architectures (JEPA), and Bidirectional Encoder Representations from Transformers (BERT). We demonstrate how it can be used to encode objects in sequential data like consecutive chess positions into compact yet meaningful representations. The basic principle of the architecture is to mask large portions of a sequence of latent states, similar to BERT and MAE. Then, we apply a lightweight Predictor to the latent representations that repairs gaps in the sequence in a lower-dimensional embedding space akin to JEPA. Our experiments in the domain of chess show that the Encoder refines the board representations such that meaningful chess concepts emerge clustered in the latent space. Furthermore, reconstructions of the masked board states show that the model is able to reason about the piece movements without relying on costly reinforcement learning methods. Lastly, we find that the resulting representation space allows for quick and intuitive dissections of chess games by observing the game path trajectories in this semantically rich space.

## Bio-PM-Guided Synthetic IMU Generation

DevFeed: [Bio-PM-Guided Synthetic IMU Generation](<https://devfeed.tech/articles/bio-pm-guided-synthetic-imu-generation-145200.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.26607>)

Author: Deepali Singh, Deepesh Parpani, Aviral Gupta

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Representation learning](<https://devfeed.tech/topics/representation-learning.md>), [Embeddings](<https://devfeed.tech/topics/embeddings.md>), [Feature Engineering](<https://devfeed.tech/topics/feature-engineering.md>), [AI Foundation Models](<https://devfeed.tech/topics/ai-foundation-models.md>), [Occam's razor machine learning](<https://devfeed.tech/topics/occam-s-razor-machine-learning.md>)

Tags: [autoregressive](<https://devfeed.tech/tags/autoregressive.md>), [classification](<https://devfeed.tech/tags/classification.md>), [classifier](<https://devfeed.tech/tags/classifier.md>), [conditional](<https://devfeed.tech/tags/conditional.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [data](<https://devfeed.tech/tags/data.md>), [decoder](<https://devfeed.tech/tags/decoder.md>), [diffusion](<https://devfeed.tech/tags/diffusion.md>), [distribution](<https://devfeed.tech/tags/distribution.md>), [embedding](<https://devfeed.tech/tags/embedding.md>), [encoder](<https://devfeed.tech/tags/encoder.md>), [f1](<https://devfeed.tech/tags/f1.md>), [feature](<https://devfeed.tech/tags/feature.md>), [generation](<https://devfeed.tech/tags/generation.md>), [gravity](<https://devfeed.tech/tags/gravity.md>), [imu](<https://devfeed.tech/tags/imu.md>), [measurement](<https://devfeed.tech/tags/measurement.md>), [metrics](<https://devfeed.tech/tags/metrics.md>), [models](<https://devfeed.tech/tags/models.md>), [raw](<https://devfeed.tech/tags/raw.md>), [synthetic](<https://devfeed.tech/tags/synthetic.md>)

### AI overview

The study generates synthetic IMU data in Bio-PM's representation space rather than directly as raw signals. It encodes 10-second windows with a frozen Bio-PM encoder and trains class-conditional autoregressive transformer and diffusion decoders; a PatchDecoder provides approximate waveform reconstruction. The diffusion model reaches 0.68 macro-F1 when trained on synthetic tokens and tested on real data. On WISDM across 51 subjects, combining synthetic-token augmentation with a 320-dimensional gravity-aware representation raises leave-one-subject-out macro-F1 from 0.729 to 0.809, with statistically significant gains concentrated in the hardest activity classes.

### Source excerpt

arXiv:2611.26607v1 Announce Type: new Abstract: Collecting and labeling inertial measurement unit (IMU) data for human activity recognition (HAR) is expensive, and models trained on one group of subjects often fail to generalize to unseen individuals. Even with balanced data, classifiers struggle to distinguish dynamically similar activity classes and benefit from targeted augmentation. We explore whether synthetic IMU data generated in the representation space of a pretrained biosignal encoder, Bio-PM \cite{b1}, can serve as a practical augmentation source. Rather than synthesizing raw signals, we leverage Bio-PM's frozen encoder to convert 10-second IMU windows into token embedding sequences and train class-conditional decoders in this space: (1) an autoregressive transformer decoder that predicts tokens sequentially, and (2) a diffusion decoder that learns the token distribution through a denoising objective. We analyze generation realism through distributional metrics and embedding-space visualizations, evaluate downstream utility through leave-one-subject-out HAR classification, and present a PatchDecoder for approximate raw waveform reconstruction. Our diffusion decoder achieves a Train-on-Synthetic-Test-on-Real (TSTR) macro-F1 of 0.68, demonstrating that generated tokens are realistic enough to train a classifier that transfers to real data. Combining synthetic-token augmentation with a 320-dimensional gravity-aware feature representation raises LOSO macro-F1 from 0.729 to 0.809 over a real-only baseline across 51 subjects on the WISDM benchmark, with statistically significant gains concentrated in the most challenging activity classes. These results demonstrate that high-fidelity generation in a pretrained encoder's representation space is achievable and practically useful for cross-subject HAR augmentation.

## Predictive Self-Supervised Learning Provably Identifies Stochastic Signals under Nuisance

DevFeed: [Predictive Self-Supervised Learning Provably Identifies Stochastic Signals under Nuisance](<https://devfeed.tech/articles/predictive-self-supervised-learning-provably-identifies-stochastic-signals-under-nuisance-144945.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2609.37789>)

Author: Fabian A. Mikulasch, Friedemann Zenke

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Self-supervised learning](<https://devfeed.tech/topics/self-supervised-learning.md>), [Representation learning](<https://devfeed.tech/topics/representation-learning.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>)

Tags: [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [self-supervised-learning](<https://devfeed.tech/tags/self-supervised-learning.md>), [signal](<https://devfeed.tech/tags/signal.md>)

### AI overview

The paper proves that predictive self-supervised learning can identify a stochastic, prediction-relevant signal despite observation-private nuisance. It attributes this to two principles: predictive mutual information maximization preserves information needed for prediction, while latent distribution matching constrains its encoding enough to make the signal identifiable. In simulations, Gaussian predictors recover the true signal up to an affine transformation, including in dynamic environments with nuisance.

### Source excerpt

arXiv:2609.37789v2 Announce Type: replace Abstract: Self-supervised learning (SSL) by predicting in latent space, without generating the input data itself, learns highly abstract, useful representations. Intuitively, this success is often attributed to its ability to discard nuisance information that is irrelevant to prediction. However, this poses a conundrum: both stochastic variation in a prediction-relevant latent signal and true nuisance make observations partly unpredictable; how could they be distinguished? Surprisingly, we prove that common SSL methods can achieve exactly this, by implicitly instantiating a latent-variable model with stochastic dynamics and observation-private nuisance. We trace their ability to recover the stochastic signal to two complementary principles: Predictive mutual information maximization ensures that representations retain the information needed for prediction, while latent distribution matching constrains how this information is encoded, thereby making the retained signal identifiable. We confirm this identifiability result in simulations for Gaussian predictors, which recover the true signal up to an affine transformation even in dynamic, nuisance-laden environments.

## One Latent, Many Tokens: Jointly Learning Compressed Embeddings for Efficient Language Diffusion

DevFeed: [One Latent, Many Tokens: Jointly Learning Compressed Embeddings for Efficient Language Diffusion](<https://devfeed.tech/articles/one-latent-many-tokens-jointly-learning-compressed-embeddings-for-efficient-language-diffusion-145381.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2609.33698>)

Author: Yulin Yuan, Ying Zhang, Xiangming Meng

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Embeddings](<https://devfeed.tech/topics/embeddings.md>), [Representation learning](<https://devfeed.tech/topics/representation-learning.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>)

Tags: [compression](<https://devfeed.tech/tags/compression.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [diffusion](<https://devfeed.tech/tags/diffusion.md>), [embeddings](<https://devfeed.tech/tags/embeddings.md>), [flow](<https://devfeed.tech/tags/flow.md>), [flow-matching](<https://devfeed.tech/tags/flow-matching.md>), [generation](<https://devfeed.tech/tags/generation.md>), [language](<https://devfeed.tech/tags/language.md>), [language-model](<https://devfeed.tech/tags/language-model.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [learning](<https://devfeed.tech/tags/learning.md>), [modeling](<https://devfeed.tech/tags/modeling.md>), [structured](<https://devfeed.tech/tags/structured.md>), [throughput](<https://devfeed.tech/tags/throughput.md>), [tokens](<https://devfeed.tech/tags/tokens.md>)

### AI overview

JPEG-DLM reduces diffusion language generation cost by jointly training a compressor, flow matching model, and decoder, so compressed embeddings are designed to be easier to model and decode. This contrasts with two-stage methods that fix the compressed embedding space before training the diffusion model, which the abstract says can hinder modeling and token recovery. The authors report the highest throughput among recent diffusion and flow models on LM1B and OWT, with Gen-PPL lower than most. On OWT at a 0.5 compression rate, JPEG-DLM reports Gen-PPL 34.52 and about 2.3x ELF's throughput.

### Source excerpt

arXiv:2609.33698v2 Announce Type: replace Abstract: Most continuous diffusion language models process one latent position per token at each sampling step, making generation expensive. Two-stage methods lower the cost by reducing the latent length, but they fix the compressed embedding space before training the diffusion model. Embeddings from the fixed space can be difficult to model with diffusion and decode reliably into tokens, which limits generation quality after compression. To address this problem, we introduce JPEG-DLM (Joint-embedding Prediction for Efficient Generation with Diffusion Language Model), which jointly trains a compressor, a flow matching model and a decoding module. With joint-embedding prediction, JPEG-DLM learns compressed embeddings that are more structured, easier to model with diffusion and reliably decodable into tokens.JPEG-DLM achieves the highest throughput among recent diffusion and flow models on LM1B and OWT, with a Gen-PPL lower than most of them. At a compression rate of 0.5 on OWT, it reaches a Gen-PPL of 34.52 and approximately 2.3 times ELF's throughput. These results suggest that jointly learning compressed embeddings offers a promising path toward efficient diffusion language modeling.Code is available at https://github.com/infiniteYuanyl/JPEG-DLM.

## Protein Functional Semantic Alignment via Representation Rectification

DevFeed: [Protein Functional Semantic Alignment via Representation Rectification](<https://devfeed.tech/articles/protein-functional-semantic-alignment-via-representation-rectification-145050.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.25396>)

Author: Chuyang Zhou, Bingxin Zhou, Andi Han, Chang Xu

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Representation learning](<https://devfeed.tech/topics/representation-learning.md>), [Embeddings](<https://devfeed.tech/topics/embeddings.md>), [post-training](<https://devfeed.tech/topics/post-training.md>), [The Gene Ontology Consortium](<https://devfeed.tech/topics/geneontology.md>)

Tags: [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [covariance](<https://devfeed.tech/tags/covariance.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [embedding](<https://devfeed.tech/tags/embedding.md>), [function](<https://devfeed.tech/tags/function.md>), [functional](<https://devfeed.tech/tags/functional.md>), [go](<https://devfeed.tech/tags/go.md>), [homology](<https://devfeed.tech/tags/homology.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [post-training](<https://devfeed.tech/tags/post-training.md>), [prediction](<https://devfeed.tech/tags/prediction.md>), [protein](<https://devfeed.tech/tags/protein.md>)

### AI overview

SAVR uses Pfam family co-membership as a homology-guided signal for contrastive post-training of protein language models, aiming to separate functional information from sequence variation. The method reduces directional dominance and variance concentration while increasing the effective dimensionality of embedding representations. It uses no task-specific labels during post-training and reports state-of-the-art results on Enzyme Commission benchmarks and competitive results on Gene Ontology benchmarks with frozen backbones. The supplied abstract does not provide benchmark details or quantify improvements.

### Source excerpt

arXiv:2611.25396v1 Announce Type: new Abstract: Protein language models (PLMs) are commonly pretrained with likelihood-based objectives such as masked language modeling or denoising. These objectives capture sequence-level regularities, but can produce representations in which functional signals are entangled with non-functional variation (e.g., sequence identity, compositional bias, and frequent local motifs), creating a readout bottleneck for downstream function prediction. We introduce Functional Semantic Alignment via Representation Rectification (SAVR), a contrastive post-training method using Pfam family co-membership as a homology-guided discrete semantic signal to reshape PLM representations. SAVR reduces directional dominance and redistributes variance across embedding dimensions, decreasing variance concentration, and increasing effective dimensionality of the embedding covariance spectrum. Without observing any task-specific labels during post-training, SAVR achieves state-of-the-art results on Enzyme Commission (EC) benchmarks and competitive results on Gene Ontology (GO) benchmarks under frozen-backbone evaluation. These results suggest that homology-guided contrastive post-training can rectify PLM representation geometry and improve sequence-based protein function prediction.

## FG-GraphBench: Benchmarking and Enhancing Fine-Grained Visual Understanding with Multimodal Scene Graphs

DevFeed: [FG-GraphBench: Benchmarking and Enhancing Fine-Grained Visual Understanding with Multimodal Scene Graphs](<https://devfeed.tech/articles/fg-graphbench-benchmarking-and-enhancing-fine-grained-visual-understanding-with-multimodal-scene-graphs-145052.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.25408>)

Author: Sicheng Liu, Xunkai Li, Ziyu Han, Xingnan Zhang, Ru Zhang, Rong-Hua Li

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [AI Foundation Models](<https://devfeed.tech/topics/ai-foundation-models.md>), [Representation learning](<https://devfeed.tech/topics/representation-learning.md>)

Tags: [benchmark](<https://devfeed.tech/tags/benchmark.md>), [benchmarking](<https://devfeed.tech/tags/benchmarking.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [fine](<https://devfeed.tech/tags/fine.md>), [generation](<https://devfeed.tech/tags/generation.md>), [graphs](<https://devfeed.tech/tags/graphs.md>), [image](<https://devfeed.tech/tags/image.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [models](<https://devfeed.tech/tags/models.md>), [multimodal](<https://devfeed.tech/tags/multimodal.md>), [pipeline](<https://devfeed.tech/tags/pipeline.md>), [qa](<https://devfeed.tech/tags/qa.md>), [stage](<https://devfeed.tech/tags/stage.md>), [tokens](<https://devfeed.tech/tags/tokens.md>)

### AI overview

The paper introduces FG-GraphBench, a benchmark for fine-grained visual understanding with tasks in visual question answering, image captioning, and image description. Its two-stage construction pipeline discovers and grounds small visual entities and their relationships in multimodal scene graphs. The authors also propose FiGRA, which uses a two-stage training strategy to encode multimodal entity and relation features as continuous graph tokens and integrate them into MLLMs. The abstract reports that FiGRA outperforms strong baselines across all three benchmark tasks, and that preserving fine-grained entity and relation information matters for visual understanding.

### Source excerpt

arXiv:2611.25408v1 Announce Type: new Abstract: Fine-grained visual understanding requires multimodal large language models (MLLMs) to recognize small visual entities and understand their relationships. Scene graphs offer a natural way to represent this information, organizing visual entities as nodes and their relationships as edges. However, we identify two challenges: existing scene graph benchmarks provide insufficient coverage of fine-grained entities and the relationships among them, and existing methods for fine-grained visual understanding have yet to effectively leverage such graph-structured information in MLLMs. To address these challenges, we introduce Fine-Grained Multimodal Scene Graph Benchmark (FG-GraphBench) and FiGRA. FG-GraphBench is a benchmark comprising three complementary tasks: fine-grained visual question answering (FG-QA), image captioning (FG-Caption), and image description generation (FG-Description). We develop a two-stage construction pipeline that discovers and grounds fine-grained entities and establishes their relations to construct the fine-grained multimodal scene graphs (MSGs) in FG-GraphBench. Building on these MSGs, we propose FiGRA, a Fine-grained Graph Representation Augmentation method for MLLMs. FiGRA adopts a two-stage training strategy. It encodes multimodal entity and relation features as continuous graph tokens, and integrates them into the MLLM. Experiments show that FiGRA significantly outperforms strong baselines across three tasks of FG-GraphBench. Further analyses confirm the effectiveness of FG-GraphBench and highlight the importance of preserving fine-grained entity and relation information for visual understanding.

## Separating Interpretability and Reconstruction in Sparse Autoencoders

DevFeed: [Separating Interpretability and Reconstruction in Sparse Autoencoders](<https://devfeed.tech/articles/separating-interpretability-and-reconstruction-in-sparse-autoencoders-145016.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.25165>)

Author: Anh Tien Nguyen, Dimitris Samaras, Mahdi S. Hosseini

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Representation learning](<https://devfeed.tech/topics/representation-learning.md>), [LLM Techniques](<https://devfeed.tech/topics/llm-techniques.md>)

Tags: [autoencoders](<https://devfeed.tech/tags/autoencoders.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [interpretability](<https://devfeed.tech/tags/interpretability.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [mechanistic-interpretability](<https://devfeed.tech/tags/mechanistic-interpretability.md>), [representation-learning](<https://devfeed.tech/tags/representation-learning.md>), [topic-modeling](<https://devfeed.tech/tags/topic-modeling.md>)

### AI overview

DualSAE separates sparse feature interpretability from reconstruction fidelity by adding a Residual Branch to a standard SAE. The Sparse Branch learns a sparse code for dominant semantics; the Residual Branch models omitted non-semantic information. A decorrelation loss limits overlap between their reconstruction contributions, and two-stage training stabilizes sparse learning before coupling the branches. Across encoder-SAE configurations on two vision datasets and two language corpora, the abstract reports mean relative gains of about 29% in reconstruction fidelity and 34% in monosemanticity score, plus consistent gains in topic modeling, hypothesis generation, and LLM steering. These are reported results from the described evaluations; the excerpt provides no detailed metrics or setup.

### Source excerpt

arXiv:2611.25165v1 Announce Type: new Abstract: Sparse Autoencoders (SAEs) are effective tools for mechanistic interpretability by decomposing neural representations into sparse and interpretable features. However, a single sparse representation in traditional SAEs is responsible for both reconstruction fidelity and interpretability, which are not guaranteed to align. To separate these objectives, we propose DualSAE, a modular framework that augments standard SAEs with a plug-in Residual Branch. A standard SAE serves as the Sparse Branch, learning a sparse code that contains the dominant semantics of the input. The added Residual Branch models the non-semantic information omitted by the Sparse Branch. We ensure that the contributions of the two branches towards the overall reconstruction have minimal overlap through a decorrelation loss. We also adopt a two-stage training strategy that first stabilizes sparse representation learning before coupling with the Residual Branch. Across diverse encoder-SAE configurations on two vision datasets and two language corpora, DualSAE achieves mean relative improvements of approximately 29% in reconstruction fidelity and 34% in interpretability (measured by monosemanticity score). Moreover, with more interpretable representations, DualSAE yields consistent performance gains in the downstream applications of topic modeling, hypothesis generation, and LLM steering. Qualitative analysis further suggests that the Sparse Branch indeed preserves dominant semantic content, whereas the Residual Branch captures non-semantic information to improve reconstruction fidelity.

## Implicit Neural Representations of Individual Behavior

DevFeed: [Implicit Neural Representations of Individual Behavior](<https://devfeed.tech/articles/implicit-neural-representations-of-individual-behavior-144906.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2606.12200>)

Author: Andrew Kang, Priya Narasimhan

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Self-supervised learning](<https://devfeed.tech/topics/self-supervised-learning.md>), [Representation learning](<https://devfeed.tech/topics/representation-learning.md>), [Neural Network](<https://devfeed.tech/topics/neural-network.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>)

Tags: [actions](<https://devfeed.tech/tags/actions.md>), [agents](<https://devfeed.tech/tags/agents.md>), [annotations](<https://devfeed.tech/tags/annotations.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [avoid](<https://devfeed.tech/tags/avoid.md>), [behavior](<https://devfeed.tech/tags/behavior.md>), [behaviors](<https://devfeed.tech/tags/behaviors.md>), [chess](<https://devfeed.tech/tags/chess.md>), [continuous](<https://devfeed.tech/tags/continuous.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [data](<https://devfeed.tech/tags/data.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [distribution](<https://devfeed.tech/tags/distribution.md>), [environments](<https://devfeed.tech/tags/environments.md>), [film](<https://devfeed.tech/tags/film.md>), [fixed](<https://devfeed.tech/tags/fixed.md>), [formula-1](<https://devfeed.tech/tags/formula-1.md>), [formula-1-racing](<https://devfeed.tech/tags/formula-1-racing.md>), [function](<https://devfeed.tech/tags/function.md>), [games](<https://devfeed.tech/tags/games.md>), [neural](<https://devfeed.tech/tags/neural.md>)

### AI overview

Behavioral INR infers policy identity from unlabeled episodes by representing each policy as a state-to-action function, modulated by an episode-level latent through FiLM layers. This self-supervised generative approach accommodates variable episode lengths and sampling granularities. The paper defines policy-level OOD shifts along state and action distribution axes and evaluates on synthetic data, MuJoCo, chess, Formula 1 racing, robotics, and Seek-Avoid datasets. The abstract reports its most consistent gains in difficult continuous state-action settings, especially with longer episodes, more policies, and OOD splits; amortized history encoders remain competitive when symbolic repetition or low-dimensional action statistics reveal identity.

### Source excerpt

arXiv:2606.12200v2 Announce Type: replace Abstract: We study policy representation learning from unlabeled multi-policy behavioral data. Each episode is generated by a fixed policy, but policy labels are unavailable. This setting appears in robotics play, demonstrations, games, racing, and other datasets where heterogeneous behaviors are mixed without annotations. We introduce Behavioral INR, a self-supervised generative model that adapts implicit neural representations (INRs) from vision to behavior. Instead of mapping coordinates to RGB values, Behavioral INR represents a policy as a state-action function mapping states to subsequent actions. An episode-level latent modulates this function through FiLM layers, yielding a generative prior over policies and allowing policy identity to be inferred without supervision. Because INRs treat each datapoint as samples from an underlying function, the same model naturally accommodates variable episode lengths and different sampling granularities, as in vision INRs with different image resolutions. We also define policy-level out-of-distribution (OOD) shifts along state-distribution and action-distribution axes, which arise when policies overlap in states or actions but are not captured by standard behavioral OOD settings based only on new agents or environments. We evaluate on synthetic Gaussian random field data, MuJoCo demonstrations with controlled OOD splits, and real-world chess, Formula 1 racing, robotics, and Seek-Avoid datasets. Behavioral INR most consistently improves policy identifiability in the hardest continuous state-action settings, especially when longer episodes, more policies, and OOD splits reduce the usefulness of marginal shortcuts; amortized history encoders remain competitive when policy identity can be recovered from symbolic repetition or low-dimensional action statistics.

## ICARE: cross-domain text classification with incremental class-aware representation and distillation learning

DevFeed: [ICARE: cross-domain text classification with incremental class-aware representation and distillation learning](<https://devfeed.tech/articles/icare-cross-domain-text-classification-with-incremental-class-aware-representation-and-distillation-learning-144992.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.24959>)

Author: Son Duong, Truong X. Tran, Hai-Anh Tran

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [text-classification](<https://devfeed.tech/topics/text-classification.md>), [Multitask learning](<https://devfeed.tech/topics/multitask-learning.md>), [Representation learning](<https://devfeed.tech/topics/representation-learning.md>), [Post-training optimization](<https://devfeed.tech/topics/post-training-optimization.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [classification](<https://devfeed.tech/tags/classification.md>), [cross](<https://devfeed.tech/tags/cross.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [distillation](<https://devfeed.tech/tags/distillation.md>), [domain](<https://devfeed.tech/tags/domain.md>), [framework](<https://devfeed.tech/tags/framework.md>), [incremental](<https://devfeed.tech/tags/incremental.md>), [inference](<https://devfeed.tech/tags/inference.md>), [knowledge-transfer](<https://devfeed.tech/tags/knowledge-transfer.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [large-language-models-llms](<https://devfeed.tech/tags/large-language-models-llms.md>), [learning](<https://devfeed.tech/tags/learning.md>), [llms](<https://devfeed.tech/tags/llms.md>), [news](<https://devfeed.tech/tags/news.md>), [overfitting](<https://devfeed.tech/tags/overfitting.md>), [pruning](<https://devfeed.tech/tags/pruning.md>), [representation-learning](<https://devfeed.tech/tags/representation-learning.md>), [sentiment-analysis](<https://devfeed.tech/tags/sentiment-analysis.md>), [text-classification](<https://devfeed.tech/tags/text-classification.md>)

### AI overview

ICARE aims to reduce catastrophic forgetting in continual text classification by combining representation-based instance selection with knowledge distillation across tasks. Its distillation transfers feature-level and logit-level knowledge using pruning techniques and optimized objectives. The abstract reports improved target accuracy and a lower forgetting metric on benchmark text classification datasets, but provides no dataset names, numerical results, or detailed comparison baselines.

### Source excerpt

arXiv:2611.24959v1 Announce Type: cross Abstract: The rise of powerful, fine-tuned large language models (LLMs) necessitates robust natural language inference (NLI) systems. These NLI systems ideally train across various tasks, encompassing diverse concepts like sentiment analysis, news classification, and coreference resolution. However, this continual, multi-tasking approach can lead to catastrophic forgetting, where previously learned tasks are lost, especially in domains lacking specific labels. Rehearsing past examples with labels can further exacerbate this issue by causing overfitting, hindering the system's ability to classify new categories. To tackle these challenges, this research proposes ICARE, a novel framework with two key components: an instance selection module and a knowledge distillation method. The former is based on representation learning, which meticulously selects training examples to present to the NLI system. The latter, knowledge distillation with feature and logit-level transfer, facilitates knowledge transfer between tasks. This leverages pruning techniques and optimized objective functions to transfer both essential feature-level and decision-making (logit-level) knowledge, mitigating catastrophic forgetting while adapting to new patterns. Extensive evaluations on benchmark text classification datasets have demonstrated ICARE's effectiveness. The framework achieves a significant improvement in target accuracy and reduction in the ``forgetting metric''. These results solidify ICARE's ability to preserve and leverage learned knowledge across diverse tasks, ultimately enhancing the adaptability and performance of NLI systems. The source code is available at https://github.com/CongSon01/ICARE_NLP.git

## SCAR: State-Space Compression for AI-Based Radio Resource Management of Vehicular Services

DevFeed: [SCAR: State-Space Compression for AI-Based Radio Resource Management of Vehicular Services](<https://devfeed.tech/articles/scar-state-space-compression-for-ai-based-radio-resource-management-of-vehicular-services-144855.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2508.06243>)

Author: Ioan-Sorin Comsa, Purav Shah, Karthik Vaidhyanathan, Deepak Gangadharan, Christof Imhof, Per Bergamin, Aryan Kaushik, Gabriel-Miro Muntean, Ramona Trestian

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [clustering](<https://devfeed.tech/topics/clustering.md>), [Representation learning](<https://devfeed.tech/topics/representation-learning.md>), [Compression](<https://devfeed.tech/topics/compression.md>), [World models](<https://devfeed.tech/topics/world-models.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [algorithms](<https://devfeed.tech/tags/algorithms.md>), [clustering](<https://devfeed.tech/tags/clustering.md>), [compression](<https://devfeed.tech/tags/compression.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-ne](<https://devfeed.tech/tags/cs-ne.md>), [cs-sy](<https://devfeed.tech/tags/cs-sy.md>), [edge](<https://devfeed.tech/tags/edge.md>), [eess-sy](<https://devfeed.tech/tags/eess-sy.md>), [fairness](<https://devfeed.tech/tags/fairness.md>), [learning](<https://devfeed.tech/tags/learning.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [management](<https://devfeed.tech/tags/management.md>), [management-policies](<https://devfeed.tech/tags/management-policies.md>), [metrics](<https://devfeed.tech/tags/metrics.md>), [ml](<https://devfeed.tech/tags/ml.md>), [network](<https://devfeed.tech/tags/network.md>), [radio](<https://devfeed.tech/tags/radio.md>), [results](<https://devfeed.tech/tags/results.md>), [rl](<https://devfeed.tech/tags/rl.md>), [sast](<https://devfeed.tech/tags/sast.md>), [services](<https://devfeed.tech/tags/services.md>), [simulation](<https://devfeed.tech/tags/simulation.md>), [state-space](<https://devfeed.tech/tags/state-space.md>)

### AI overview

SCAR compresses high-dimensional vehicular network Channel Quality Indicator data into compact states for reinforcement-learning-based radio resource management. Its edge-assisted framework uses clustering and radial basis function networks to preserve information relevant to decisions, allowing policies to adjust the throughput-fairness trade-off against NGMN service-level fairness objectives. In simulations, compression raised average occupancy of the NGMN feasible region from 56.32% to 88.07% across ten RL algorithms, with the best configuration reaching 97.76%; the abstract reports limited throughput degradation and competitive computational overhead. SAST-based clustering reduced compression distortion by about 10% versus conventional K-means. These findings are simulation results, not evidence of deployment performance.

### Source excerpt

arXiv:2508.06243v3 Announce Type: replace Abstract: The increasing demand for connected vehicular services poses significant challenges for AI-based network and service management due to the high volume and rapid variability of network state information. Traditional management and control mechanisms struggle to scale when processing fine-grained metrics such as Channel Quality Indicators (CQIs) in dynamic vehicular environments. To address this challenge, we propose SCAR (State-Space Compression for AI-Based Radio Resource Management), an edge-assisted framework that abstracts high-dimensional CQI information into compact representations for AI-based management. SCAR employs machine-learning (ML)-based compression techniques, including clustering and radial basis function (RBF) networks, to reduce the dimensionality of CQI-derived state information while preserving essential features relevant to management decisions. The resulting compressed states are used to train reinforcement learning (RL)-based management policies that dynamically adapt the throughput-fairness trade-off according to service-level fairness objectives defined by the NGMN. Simulation results show that CQI compression improves average NGMN feasible-region occupancy from 56.32% to 88.07% across ten RL algorithms, reaching 97.76% for the best-performing RL configuration, with limited throughput degradation and competitive computational overhead. Furthermore, simulated annealing with stochastic tunneling (SAST)-based clustering reduces state compression distortion by approximately 10% compared with conventional K-means clustering. These results demonstrate SCAR's ability to provide compact state representations for RL-based network and service management while improving service-level fairness in dynamic vehicular systems.

## A Neurosymbolic Multimodal Framework for 30-Day Diabetic Readmission Prediction: Integrating Structured EHR Data, Dense Clinical Embeddings, and Knowledge-Grounded Symbolic Extraction

DevFeed: [A Neurosymbolic Multimodal Framework for 30-Day Diabetic Readmission Prediction: Integrating Structured EHR Data, Dense Clinical Embeddings, and Knowledge-Grounded Symbolic Extraction](<https://devfeed.tech/articles/a-neurosymbolic-multimodal-framework-for-30-day-diabetic-readmission-prediction-integrating-structured-ehr-data-dense-clinical-embeddings-and-knowledge-grounded-symbolic-145488.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.25236>)

Author: Una Suman Kumar Patro, Sibangi Subhadarsani, Narendra Badam

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Vectors & LLMs](<https://devfeed.tech/topics/vectors-llms.md>), [Embeddings](<https://devfeed.tech/topics/embeddings.md>), [Representation learning](<https://devfeed.tech/topics/representation-learning.md>), [AI Integration Strategies](<https://devfeed.tech/topics/ai-integration-strategies.md>), [AI Chat](<https://devfeed.tech/topics/ai-chat.md>), [AI for data automation](<https://devfeed.tech/topics/ai-for-data-automation.md>), [data-engineering](<https://devfeed.tech/topics/data-engineering.md>)

Tags: [cross-encoder](<https://devfeed.tech/tags/cross-encoder.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [dataset](<https://devfeed.tech/tags/dataset.md>), [embeddings](<https://devfeed.tech/tags/embeddings.md>), [hallucinations](<https://devfeed.tech/tags/hallucinations.md>), [multimodal](<https://devfeed.tech/tags/multimodal.md>), [retrieval-augmented-generation](<https://devfeed.tech/tags/retrieval-augmented-generation.md>), [structured](<https://devfeed.tech/tags/structured.md>), [xgboost](<https://devfeed.tech/tags/xgboost.md>)

### AI overview

On a held-out cohort of 20,153 encounters, the multimodal model improved sensitivity over the structured EHR baseline (0.6667 vs. 0.5839), achieved the best F2-score (0.3824) and lower Brier score (0.0977 vs. 0.1080), but had slightly lower AUROC (0.6375 vs. 0.6477) and specificity (0.5161). It combines engineered structured features, 32 compressed Bio_ClinicalBERT embedding components, and symbolic entities extracted through hybrid retrieval and negation auditing, then predicts with cost-sensitive XGBoost. In a separate 500-note code-switched benchmark, grounded retrieval raised Hinglish symptom recall and precision and reduced entity hallucinations. Prior utilization was the dominant risk signal. The reported findings are from the abstract's dataset and evaluations.

### Source excerpt

arXiv:2611.25236v1 Announce Type: new Abstract: Unplanned 30-day readmissions among patients with diabetes mellitus impose a substantial clinical and economic burden, yet most predictive models use only structured electronic health record (EHR) data and ignore free-text clinical notes, which often contain code-mixed Hindi-English (Hinglish) phrasing and negation that general-purpose language models parse unreliably. We propose a neurosymbolic multimodal framework evaluated on the public Diabetes 130-US Hospitals dataset (101,766 encounters; 71,518 patients) with a patient-level grouped 80/20 split. Three streams are fused: engineered structured features; 32 components of Bio_ClinicalBERT note embeddings obtained by truncated singular value decomposition; and symbolic entities extracted by an open-weight clinical language model within a three-stage hybrid retrieval-augmented generation pipeline (dense retrieval, sparse lexical retrieval, and cross-encoder re-ranking) with reflective negation auditing. A cost-sensitive XGBoost classifier with a decision threshold calibrated by Youden's J statistic produces the prediction. On the held-out test cohort (n = 20,153), the structured baseline achieved an area under the receiver operating characteristic curve (AUROC) of 0.6477, sensitivity of 0.5839, and Brier score of 0.1080. The full multimodal model achieved the highest sensitivity (0.6667; +8.28 percentage points), the best F2-score (0.3824), and the lowest Brier score (0.0977), at a slightly lower AUROC (0.6375) and specificity (0.5161). Shapley additive explanations showed that prior utilization dominates risk, with compressed narrative components among the top predictors. On a 500-note code-switched benchmark, grounded retrieval raised Hinglish symptom recall from 4.8% to 40.2% and precision from 3.1% to 45.2%, while reducing entity hallucinations from 1.74 to 0.00 per note.

## C-LiTE: A Lightweight Foundation Model for Cellular feature learning in Histopathology

DevFeed: [C-LiTE: A Lightweight Foundation Model for Cellular feature learning in Histopathology](<https://devfeed.tech/articles/c-lite-a-lightweight-foundation-model-for-cellular-feature-learning-in-histopathology-145653.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.26316>)

Author: Esha Sadia Nasir, Jiaqi Lv, Gozde N. Gunesli, Shan E Ahmed Raza

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [AI Foundation Models](<https://devfeed.tech/topics/ai-foundation-models.md>), [Representation learning](<https://devfeed.tech/topics/representation-learning.md>), [Training AI Models](<https://devfeed.tech/topics/training-ai-models.md>), [Dataset development](<https://devfeed.tech/topics/dataset-development.md>), [image recognition](<https://devfeed.tech/topics/image-recognition.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>)

Tags: [cellular](<https://devfeed.tech/tags/cellular.md>), [classification](<https://devfeed.tech/tags/classification.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [embedding](<https://devfeed.tech/tags/embedding.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [f1](<https://devfeed.tech/tags/f1.md>), [features](<https://devfeed.tech/tags/features.md>), [fewer](<https://devfeed.tech/tags/fewer.md>), [figure](<https://devfeed.tech/tags/figure.md>), [foundation-model](<https://devfeed.tech/tags/foundation-model.md>), [lightweight](<https://devfeed.tech/tags/lightweight.md>), [native](<https://devfeed.tech/tags/native.md>), [public](<https://devfeed.tech/tags/public.md>), [representation-learning](<https://devfeed.tech/tags/representation-learning.md>), [token](<https://devfeed.tech/tags/token.md>), [training-dataset](<https://devfeed.tech/tags/training-dataset.md>), [transformation](<https://devfeed.tech/tags/transformation.md>), [type](<https://devfeed.tech/tags/type.md>), [vit](<https://devfeed.tech/tags/vit.md>)

### AI overview

C-LiTE is a 22-million-parameter model for learning cellular features from H&E histopathology images. It processes 64x64-pixel cell-centred crops at native resolution, using 4x4-pixel patches to form a 16x16 token grid without upsampling; its patch embedding is initialized by adapting weights from a pre-trained ViT-5-S with a pseudoinverse-based transformation. The model is pre-trained on 60 million TCGA cell crops balanced across 19 cancer types and 16 tissue types. Across seven public datasets for mitotic figure recognition, nucleus and cell-type classification, and immune-cell phenotyping, frozen-feature evaluation yields mean macro-F1 0.670, close to CytoFormer's 0.682 with about 30 times fewer parameters. These findings are limited to the reported evaluation setting.

### Source excerpt

arXiv:2611.26316v1 Announce Type: cross Abstract: Characterising individual cells is fundamental to computational pathology, as cellular morphology informs tasks such as tumour grading, mitotic assessment and immune profiling. However, most pathology foundation models are trained on tissue patches and capture cellular features only implicitly, while existing cell-level models are either restricted to specific cell populations or rely on computationally expensive architectures with hundreds of millions of parameters. Applying ViTs directly to small cellular crops also presents an architectural challenge: upsampling increases computational cost without adding morphological information and may introduce interpolation artefacts. To address these limitations, we introduce C-LiTE, a lightweight foundation model for cellular representation learning in H&E-stained histopathology, with only 22 million parameters. C-LiTE employs a native-resolution tokenisation strategy tailored to cellular inputs, processing 64 x 64 px cell-centred crops using a 4 x 4 patch embedding to produce a 16 x 16 token grid without image upsampling. To transfer visual knowledge from large-scale pre-training, we adapt the patch embedding weights of a pre-trained ViT-5-S using a pseudoinverse-based transformation, enabling weight initialisation at the finer patch resolution. We further assemble a pre-training dataset of 60 million cell-centred crops from TCGA, balanced across 19 cancer types and 16 tissue types. To our knowledge, this represents the largest public dataset for self-supervised pre-training on H&E images at the cellular level. We evaluate C-LiTE on seven public datasets spanning mitotic figure recognition, nucleus and cell-type classification, and immune cell phenotyping. Under frozen-feature evaluation, C-LiTE achieves a mean macro-F1 of 0.670, approaching CytoFormer (0.682) with approximately 30x fewer parameters.

## Learning to Compare Neural Representations from Parameterized Computation Graphs

DevFeed: [Learning to Compare Neural Representations from Parameterized Computation Graphs](<https://devfeed.tech/articles/learning-to-compare-neural-representations-from-parameterized-computation-graphs-145137.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.26081>)

Author: Ya-Wei Eileen Lin, Daniel Herbst, Stefanie Jegelka

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Representation learning](<https://devfeed.tech/topics/representation-learning.md>), [Neural Network](<https://devfeed.tech/topics/neural-network.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [model architecture](<https://devfeed.tech/topics/model-architecture.md>), [generalization in machine learning](<https://devfeed.tech/topics/generalization-in-machine-learning.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [accounting](<https://devfeed.tech/tags/accounting.md>), [adapter](<https://devfeed.tech/tags/adapter.md>), [also](<https://devfeed.tech/tags/also.md>), [analyze](<https://devfeed.tech/tags/analyze.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [attention](<https://devfeed.tech/tags/attention.md>), [basis](<https://devfeed.tech/tags/basis.md>), [can](<https://devfeed.tech/tags/can.md>), [candidate](<https://devfeed.tech/tags/candidate.md>), [collecting](<https://devfeed.tech/tags/collecting.md>), [collection](<https://devfeed.tech/tags/collection.md>), [compare](<https://devfeed.tech/tags/compare.md>), [comparisons](<https://devfeed.tech/tags/comparisons.md>), [components](<https://devfeed.tech/tags/components.md>), [conditional](<https://devfeed.tech/tags/conditional.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [embedding](<https://devfeed.tech/tags/embedding.md>), [graphs](<https://devfeed.tech/tags/graphs.md>), [learning](<https://devfeed.tech/tags/learning.md>), [neural](<https://devfeed.tech/tags/neural.md>)

### AI overview

The paper proposes predicting similarity between internal representations in unseen models from their parameterized computations, avoiding the need to first compare activations on reference inputs. A graph metanetwork embeds each internal site from its upstream computation and accounts for hidden-coordinate permutations and internal attention-basis transformations. The authors report relation estimation, low-regret layer retrieval, and selection of useful stitching connections, even though the predictor was not trained on stitching outcomes. They derive conditional guarantees for retrieval and stitching candidate selection. The abstract frames predictions as a way to guide candidates before direct measurement or adapter fitting; it does not provide detailed experimental settings or results.

### Source excerpt

arXiv:2611.26081v1 Announce Type: new Abstract: Comparing internal representations across trained models is important for identifying corresponding layers and selecting components for model reuse. Yet, direct comparisons at inference require evaluating candidate models on known reference inputs and collecting their internal activations. In this paper, we study whether internal representation similarity measured across independently trained models can instead be learned to predict in unseen models from their parameterized computations. We introduce a framework that applies a graph metanetwork for mapping each internal site to an embedding derived from its upstream computations while accounting for hidden-coordinate permutation and internal attention-basis transformations. We analyze the structural properties of our estimators and derive conditional guarantees for layer retrieval and stitching candidate selection. Empirically, we find that the learned predictors support relation estimation and low-regret layer retrieval. They also select useful stitching connections while not being trained on stitching outcomes. This opens up a new way to study representation similarity directly from trained neural networks: representational relations learned across a collection of trained models can generalize to new computations, and that the predicted relations can guide candidate selection before direct measurement or adapter fitting for functional evaluations.

## Persistent Sparse Autoencoders: Learning Feature-Specific Timescales in Language Model Representations

DevFeed: [Persistent Sparse Autoencoders: Learning Feature-Specific Timescales in Language Model Representations](<https://devfeed.tech/articles/persistent-sparse-autoencoders-learning-feature-specific-timescales-in-language-model-representations-144914.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2607.17117>)

Author: Haoyan Luo, Mateo Espinosa Zarlenga, Mateja Jamnik

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Representation learning](<https://devfeed.tech/topics/representation-learning.md>), [Language models](<https://devfeed.tech/topics/language-models.md>), [Neural Network](<https://devfeed.tech/topics/neural-network.md>), [LLM observability](<https://devfeed.tech/topics/llm-observability.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [autoencoders](<https://devfeed.tech/tags/autoencoders.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [error](<https://devfeed.tech/tags/error.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [extension](<https://devfeed.tech/tags/extension.md>), [fast](<https://devfeed.tech/tags/fast.md>), [feature](<https://devfeed.tech/tags/feature.md>), [features](<https://devfeed.tech/tags/features.md>), [hidden](<https://devfeed.tech/tags/hidden.md>), [hidden-states](<https://devfeed.tech/tags/hidden-states.md>), [information](<https://devfeed.tech/tags/information.md>), [language-model](<https://devfeed.tech/tags/language-model.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [learning](<https://devfeed.tech/tags/learning.md>), [monitor](<https://devfeed.tech/tags/monitor.md>), [monitoring](<https://devfeed.tech/tags/monitoring.md>), [persistence](<https://devfeed.tech/tags/persistence.md>), [prompt-injection](<https://devfeed.tech/tags/prompt-injection.md>), [quality](<https://devfeed.tech/tags/quality.md>), [reduce](<https://devfeed.tech/tags/reduce.md>)

### AI overview

Persistent Sparse Autoencoders (Persistent SAEs) learn a persistence coefficient for each sparse feature, giving features different timescales based on reconstruction. The authors report that standard SAE features can already persist: after activation, hidden states remain aligned with a feature direction, and past activations help reconstruct later states. Persistent SAEs retain competitive reconstruction quality while producing fast features that remain locally interpretable and slow features that accumulate higher-level context. In a prompt-injection monitoring case study, slow features retain injection signals over long contexts and causally affect monitor judgements. The abstract presents these results as opportunities for model interpretation and monitoring.

### Source excerpt

arXiv:2607.17117v3 Announce Type: replace Abstract: Sparse autoencoders (SAEs) decompose language model activations into sparse features, yet these models traditionally encode each token independently, failing to expose information that persists across a sequence. We first show that temporal persistence can naturally emerge in standard SAE features: after a feature activates, the hidden state remains aligned with its direction, and past activations reduce the reconstruction error of later hidden states. How long this lasts varies widely across features. We therefore introduce Persistent Sparse Autoencoders (Persistent SAEs), an extension of standard SAEs that learns a persistence coefficient for each feature, allowing feature-specific timescales to emerge from reconstruction. Our experiments show that Persistent SAEs retain competitive reconstruction quality while learning a spectrum of timescales: short-timescale (fast) features remain locally interpretable, whereas long-timescale (slow) features accumulate higher-level contextual information. Moreover, we show in a prompt-injection monitoring case study that slow features retain injection signals over long contexts and causally affect monitor judgements. These results suggest that Persistent SAEs offer new opportunities for interpreting and monitoring language models via persistent features.

## SplitReAlign: Partition-Aware Representation Realignment for Split Federated Unlearning

DevFeed: [SplitReAlign: Partition-Aware Representation Realignment for Split Federated Unlearning](<https://devfeed.tech/articles/splitrealign-partition-aware-representation-realignment-for-split-federated-unlearning-145038.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.25301>)

Author: Jia Chen, Yecheng Lv, Xinrong Hu, Jun Bai

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Cryptography and Security](<https://devfeed.tech/sources/arxiv-cryptography-and-security.md>), [arXiv: Distributed Computing](<https://devfeed.tech/sources/arxiv-distributed-computing.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Federated Learning](<https://devfeed.tech/topics/federated-learning.md>), [Representation learning](<https://devfeed.tech/topics/representation-learning.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [all](<https://devfeed.tech/tags/all.md>), [anchor](<https://devfeed.tech/tags/anchor.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [changes](<https://devfeed.tech/tags/changes.md>), [cifar-10](<https://devfeed.tech/tags/cifar-10.md>), [class](<https://devfeed.tech/tags/class.md>), [client](<https://devfeed.tech/tags/client.md>), [clients](<https://devfeed.tech/tags/clients.md>), [coupling](<https://devfeed.tech/tags/coupling.md>), [cs-cr](<https://devfeed.tech/tags/cs-cr.md>), [cs-dc](<https://devfeed.tech/tags/cs-dc.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [distributions](<https://devfeed.tech/tags/distributions.md>), [effectiveness](<https://devfeed.tech/tags/effectiveness.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [feature](<https://devfeed.tech/tags/feature.md>), [federated](<https://devfeed.tech/tags/federated.md>), [federated-learning](<https://devfeed.tech/tags/federated-learning.md>), [fixed](<https://devfeed.tech/tags/fixed.md>), [partition](<https://devfeed.tech/tags/partition.md>), [split](<https://devfeed.tech/tags/split.md>)

### AI overview

SplitReAlign addresses class-wise forgetting in split federated learning while aiming to preserve retained utility across heterogeneous client and server representations. It builds endpoint-specific discriminative subspaces around a shared semantic anchor, realigns forgotten representations, and penalizes changes outside those subspaces. Its update strategies reflect partition constraints: Client-only updates use gradients through frozen servers, Server-only guidance uses fixed client-feature statistics, and Joint updates optimize both partitions. On CIFAR-10 and CIFAR-100 with heterogeneous client distributions, it had the lowest average gap to retraining among compared methods across six settings; Server-only was closest to retraining on both datasets. The supplied text reports these results but does not provide numerical gaps or further experimental details.

### Source excerpt

arXiv:2611.25301v1 Announce Type: new Abstract: Split federated learning (SFL) distributes neural networks across clients and a server, coupling class-specific knowledge across their representations. This coupling complicates split federated unlearning (SFU), where class-wise forgetting must remove target knowledge while preserving retained utility under heterogeneous representation spaces and asymmetric update constraints. We propose SplitReAlign, a partition-aware representation realignment framework for class-wise SFU. It constructs endpoint-specific discriminative subspaces coordinated through a shared semantic anchor, selectively realigning forgotten representations while penalizing changes outside the active subspaces. To accommodate asymmetric update constraints, SplitReAlign leverages gradients through frozen servers for Client-only updates and fixed client-feature statistics for Server-only guidance, while Joint updates optimize both partitions. Experiments on CIFAR-10 and CIFAR-100 under heterogeneous client distributions demonstrate that SplitReAlign achieves the lowest average gap to retraining among all compared methods across six settings. Notably, Server-only achieves the closest agreement with retraining on both datasets, highlighting the effectiveness of partition-aware guidance under restricted updates.

## Omni-modal decomposition autoencoders learn full-stack wearable disentangled representations

DevFeed: [Omni-modal decomposition autoencoders learn full-stack wearable disentangled representations](<https://devfeed.tech/articles/omni-modal-decomposition-autoencoders-learn-full-stack-wearable-disentangled-representations-144917.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2608.07385>)

Author: Ioannis Ziogas, Ensieh Khazaei, Bilal Taha, Aamna Al Shehhi, Ahsan H. Khandoker, Leontios J. Hadjileontiadis, Dimitrios Hatzinakos

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Representation learning](<https://devfeed.tech/topics/representation-learning.md>), [VAE](<https://devfeed.tech/topics/vae.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Self-supervised learning](<https://devfeed.tech/topics/self-supervised-learning.md>)

Tags: [autoencoders](<https://devfeed.tech/tags/autoencoders.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [eess-as](<https://devfeed.tech/tags/eess-as.md>), [eess-sp](<https://devfeed.tech/tags/eess-sp.md>), [full-stack](<https://devfeed.tech/tags/full-stack.md>), [fusion](<https://devfeed.tech/tags/fusion.md>), [generation](<https://devfeed.tech/tags/generation.md>), [series](<https://devfeed.tech/tags/series.md>), [stat-ml](<https://devfeed.tech/tags/stat-ml.md>), [vae](<https://devfeed.tech/tags/vae.md>), [wearable](<https://devfeed.tech/tags/wearable.md>)

### AI overview

Omni-modal Variational Decomposition Autoencoders (OmniDecVAEs) use a shared encoder-decoder to learn representations across heterogeneous wearable time-series modalities. They extend DecVAEs with per-modality time-frequency latent subspaces, a multi-view self-supervised decomposition loss, and a component-wise conditional decoder. On HARWE, UCI HAR, and WESAD, the abstract reports up to 3.0 percentage points higher DCI disentanglement for unseen subjects. On HARWE, results improve subject-dependent activity and identity recognition by 1.1 and 5.8 points; on unseen subjects in multi-device datasets, they exceed supervised end-to-end models by 31.6 and 22.3 points. Reconstruction error is 34-74% lower than with multi-modal VAEs, and synthetic data improve unseen-subject affect recognition by 2 points. The model has 4.1 million parameters, with parameter count reported as invariant to modality count and 21 times fewer parameters than a multi-branch VAE.

### Source excerpt

arXiv:2608.07385v2 Announce Type: replace Abstract: Learning disentangled representations is key for general-purpose multi-modal wearable models, yet existing approaches rarely combine classification, disentanglement, fusion, and generation of heterogeneous multi-modal time series in a single model. We introduce Omni-modal Variational Decomposition Autoencoders (OmniDecVAEs), a framework that learns multi-purpose representations from arbitrarily many modalities with a single shared encoder-decoder. OmniDecVAEs extend DecVAEs by learning time-frequency latent subspaces per modality through a multi-view self-supervised decomposition loss and a component-wise conditional decoder. On the omni-modal human activity recognition (HAR) HARWE dataset (up to thirty channels) and the public UCI HAR and WESAD datasets, OmniDecVAEs learn the most disentangled representations for unseen subjects (DCI disentanglement up to 3.0 percentage points (pp) higher). On HARWE, they improve subject-dependent HAR and identity recognition by 1.1 pp and 5.8 pp, and outperform supervised end-to-end models by +31.6 pp and +22.3 pp on unseen subjects of multi-device datasets. Moreover, their generative capacity improves reconstruction mean squared error over multi-modal VAEs (34-74% improvement) and their synthetic data improve affect recognition on unseen subjects (2 pp). With 4.1M parameters, invariant to the number of modalities (21xfewer than a multi-branch VAE), OmniDecVAEs are scalable, single-model full-stack learners for wearable representations.

## Actor-Predictor Architecture for Reinforcement Learning with Formal Specification Objectives

DevFeed: [Actor-Predictor Architecture for Reinforcement Learning with Formal Specification Objectives](<https://devfeed.tech/articles/actor-predictor-architecture-for-reinforcement-learning-with-formal-specification-objectives-145652.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.26313>)

Author: Josef Brozovich, Qining Zhang, Lei Ying

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>), [Representation learning](<https://devfeed.tech/topics/representation-learning.md>), [concept-learning](<https://devfeed.tech/topics/concept-learning.md>), [grpo](<https://devfeed.tech/topics/grpo.md>), [Isaac](<https://devfeed.tech/topics/isaac.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [actor](<https://devfeed.tech/tags/actor.md>), [approach](<https://devfeed.tech/tags/approach.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [features](<https://devfeed.tech/tags/features.md>), [formal](<https://devfeed.tech/tags/formal.md>), [framework](<https://devfeed.tech/tags/framework.md>), [grid](<https://devfeed.tech/tags/grid.md>), [grpo](<https://devfeed.tech/tags/grpo.md>), [isaac-lab](<https://devfeed.tech/tags/isaac-lab.md>), [learning](<https://devfeed.tech/tags/learning.md>), [logic](<https://devfeed.tech/tags/logic.md>), [optimized](<https://devfeed.tech/tags/optimized.md>), [progress](<https://devfeed.tech/tags/progress.md>), [regular](<https://devfeed.tech/tags/regular.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [reward](<https://devfeed.tech/tags/reward.md>), [robotic](<https://devfeed.tech/tags/robotic.md>), [specification](<https://devfeed.tech/tags/specification.md>)

### AI overview

Actor-Predictor learns logic progress directly from partial trajectories, avoiding explicit automaton construction for reinforcement learning with formal specifications. A predictor estimates whether a partial trajectory will later satisfy the specification; its learned features become auxiliary actor inputs, and the actor is optimized with GRPO and RWR. The authors show that the representation is equivalent to an automaton state when predictive accuracy is sufficient. In grid-world and Isaac Lab robotic tasks, they report better performance than automaton-based baselines on LTL-constrained tasks, and performance on non-regular specifications such as $(a^n b^n)$, which finite-automaton approaches cannot handle.

### Source excerpt

arXiv:2611.26313v1 Announce Type: new Abstract: One fundamental challenge in reinforcement learning (RL) is enforcing temporal safety constraints, such as satisfying formal specifications. Existing methods often transform logic formulas into automata for state augmentation and reward shaping. However, this approach assumes that automaton construction is both feasible and tractable, which limits its applicability to small specifications and linear temporal logic (LTL). This paper proposes Actor-Predictor, a framework that learns a surrogate representation of logic progress without explicitly constructing an automaton. The predictor continuously estimates whether a partial trajectory will satisfy the formal specification in a later step, and its learned features are provided as auxiliary inputs to the actor, which is then optimized using GRPO and RWR. We show theoretically that, under sufficient predictive accuracy, the learned representation is equivalent to the automaton state. Experiments on grid world and robotic tasks in Isaac Lab demonstrate that Actor-Predictor outperforms automaton-based baselines on LTL-constrained tasks. It also performs well on non-regular formal specifications, such as $(a^n b^n)$, for which finite-automaton-based approaches are not applicable.

## Error Analysis of Deep Learning

DevFeed: [Error Analysis of Deep Learning](<https://devfeed.tech/articles/error-analysis-of-deep-learning-143375.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.23321>)

Author: Yuling Jiao, Peiying Wu, Jerry Zhijian Yang, Pingwen Zhang

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Deep learning](<https://devfeed.tech/topics/deep-learning.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>), [Representation learning](<https://devfeed.tech/topics/representation-learning.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>)

Tags: [approximation](<https://devfeed.tech/tags/approximation.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-na](<https://devfeed.tech/tags/cs-na.md>), [deep-learning](<https://devfeed.tech/tags/deep-learning.md>), [error](<https://devfeed.tech/tags/error.md>), [generative](<https://devfeed.tech/tags/generative.md>), [guarantees](<https://devfeed.tech/tags/guarantees.md>), [large-language-models](<https://devfeed.tech/tags/large-language-models.md>), [learning-theory](<https://devfeed.tech/tags/learning-theory.md>), [math-na](<https://devfeed.tech/tags/math-na.md>), [numerical-analysis](<https://devfeed.tech/tags/numerical-analysis.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [regression](<https://devfeed.tech/tags/regression.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [representation-learning](<https://devfeed.tech/tags/representation-learning.md>)

### AI overview

The review organizes deep learning guarantees around approximation, statistical, and optimization errors, tracing work from bounds on individual components to analyses that account for all three. It uses nonparametric regression and the Deep Ritz Method to illustrate the framework across different loss structures, then surveys applications in scientific machine learning, generative learning, reinforcement learning, representation learning, and large language models. Its focus is on identifying which errors are controlled, how they combine, and the assumptions required for guarantees about trained networks.

### Source excerpt

arXiv:2611.23321v1 Announce Type: cross Abstract: This review examines deep learning theory through the lens of error decomposition, systematically addressing approximation, statistical, and optimization errors. We trace the development of error analyses from bounds on individual error components to frameworks that account for all three. Using nonparametric regression and the Deep Ritz Method as dual running examples, we demonstrate how these analyses apply across fundamentally different loss structures. We then review representative analyses in scientific machine learning, generative learning, reinforcement learning, representation learning, and large language models, showing how learning errors affect solutions, distributions, and predictions through problem-specific arguments. Throughout, we clarify which error components are controlled, how they are combined, and what assumptions are needed to obtain guarantees for trained networks.

## What competing constraints reveal about representation learning

DevFeed: [What competing constraints reveal about representation learning](<https://devfeed.tech/articles/what-competing-constraints-reveal-about-representation-learning-142522.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2605.15995>)

Author: Gwenol\\'e Quellec

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Representation learning](<https://devfeed.tech/topics/representation-learning.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>)

Tags: [coherence](<https://devfeed.tech/tags/coherence.md>), [compatibility](<https://devfeed.tech/tags/compatibility.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [designing](<https://devfeed.tech/tags/designing.md>), [diagnostics](<https://devfeed.tech/tags/diagnostics.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [information](<https://devfeed.tech/tags/information.md>), [invariance](<https://devfeed.tech/tags/invariance.md>), [learning](<https://devfeed.tech/tags/learning.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [multimodal](<https://devfeed.tech/tags/multimodal.md>), [objectives](<https://devfeed.tech/tags/objectives.md>), [organization](<https://devfeed.tech/tags/organization.md>), [performance](<https://devfeed.tech/tags/performance.md>), [representation-learning](<https://devfeed.tech/tags/representation-learning.md>), [temporal](<https://devfeed.tech/tags/temporal.md>)

### AI overview

The paper frames representation learning as balancing six properties: predictive sufficiency, minimality, temporal coherence, observation compatibility, nuisance invariance, and structural constraints. It separates these desired properties from training objectives and evaluation diagnostics, emphasizing that neither guarantees the property is achieved. Controlled synthetic experiments find that favorable diagnostics can misrepresent retained information, and that equivalent predictive performance can coexist with different latent coordinates. The authors argue that complementary evaluations help interpret representations and guide their design and comparison.

### Source excerpt

arXiv:2605.15995v5 Announce Type: replace-cross Abstract: Learning latent states from temporal, multimodal, and partially observed data requires decisions about what information representations should retain, discard, and organize. I argue that six complementary properties help clarify these decisions: predictive sufficiency, minimality, temporal coherence, observation compatibility, nuisance invariance, and structural constraints. Drawing on established approaches, I distinguish desired representation properties from the objectives used to encourage them and the diagnostics used to assess them, since neither a training objective nor an evaluation metric guarantees that a property has been achieved. Controlled synthetic experiments examine how training objectives influence predictive performance, nuisance decodability, and latent organization. The results reveal discrepancies between seemingly favorable diagnostics and the information retained in learned representations, as well as latent-coordinate ambiguities under equivalent predictive performance, illustrating how complementary evaluations can improve interpretation and guide design choices. Together, these analyses provide a basis for designing, comparing, and evaluating learned latent states.

## Learning Trajectories Leave Decodable Traces in Neural Network Weight Space and Internal Representations

DevFeed: [Learning Trajectories Leave Decodable Traces in Neural Network Weight Space and Internal Representations](<https://devfeed.tech/articles/learning-trajectories-leave-decodable-traces-in-neural-network-weight-space-and-internal-representations-142831.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.24782>)

Author: Patrick Krauss, Achim Schilling, Andreas Maier, Thomas Kinfe, Claus Metzner

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Neural Network](<https://devfeed.tech/topics/neural-network.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Representation learning](<https://devfeed.tech/topics/representation-learning.md>)

Tags: [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [neural-networks](<https://devfeed.tech/tags/neural-networks.md>), [order](<https://devfeed.tech/tags/order.md>), [q-bio-nc](<https://devfeed.tech/tags/q-bio-nc.md>), [weight](<https://devfeed.tech/tags/weight.md>)

### AI overview

Neural networks trained on the same task retain selective traces of how they learned, even when they reach distinct but similarly successful solutions. In the reported experiments, all 30 trained networks were matched to their initial weights among ten candidates per dataset, and a linear probe decoded the order of two earlier tasks with 100% accuracy on 80 held-out networks after a shared final task. Earlier intermediate weights were partly predictable, but initial weights could not be reliably reconstructed. Perturbed networks recovered classification performance without retracing their original paths. Of 2,160 aligned endpoint pairs, 2,157 had no positive loss barrier at sampled interpolation points, suggesting low-loss connections between solutions rather than convergence to a single weight vector. These findings indicate that final weights preserve some historical information, not a complete record of training.

### Source excerpt

arXiv:2611.24782v1 Announce Type: cross Abstract: Neural networks trained on the same task develop related weight-space structures, yet remain distinguishable after learning. Do these differences merely reflect alternative solutions, or do they preserve information about individual learning histories? We tested whether final weights reveal initialization, task order, and earlier trajectory states. Training reorganized the ensemble: initial pairwise distances did not consistently predict final distances. Nevertheless, all 30 trained networks were correctly matched to their initial weights among ten candidates per dataset. Networks sharing an initialization also developed recognizable final hidden representations across different batch schedules, although direct comparisons with initial activity did not recover identity. Learning order remained readable as well. A linear probe decoded the order of two earlier tasks with 100% accuracy on 80 held-out networks after ten epochs of a common final task. Earlier intermediate weight states were partly predictable, but individual initial weights could not be reliably reconstructed. Final weights thus retain selective historical information rather than a complete trajectory record. Perturbed networks recovered classification performance without returning to their unperturbed training paths. This separation persisted across datasets, intervention times, and network layers. Moreover, 2,157 of 2,160 aligned endpoint pairs showed no positive loss barrier at sampled interpolation points, indicating low-loss connections between distinct solutions rather than convergence to one weight vector. Together, these findings show that a shared task constrains networks to related successful solutions without erasing their individual histories. Final weights reflect not only what a network has learned, but also where learning began and how it unfolded.

## PGU-Decoder: Prior-Guided Unified Decoding for Industrial Road-Network Map Updating

DevFeed: [PGU-Decoder: Prior-Guided Unified Decoding for Industrial Road-Network Map Updating](<https://devfeed.tech/articles/pgu-decoder-prior-guided-unified-decoding-for-industrial-road-network-map-updating-143464.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.23936>)

Author: Yuguang Ning, Bingbing Kang, Nan Zhao, Zhixuan Li, Ze Zhao, Dongdong Liu, Wenjian Zhu, Zekai Liu, Junhui Wang

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Neural Network](<https://devfeed.tech/topics/neural-network.md>), [Representation learning](<https://devfeed.tech/topics/representation-learning.md>), [autonomous vehicles](<https://devfeed.tech/topics/autonomous-vehicles.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [attention](<https://devfeed.tech/tags/attention.md>), [automated](<https://devfeed.tech/tags/automated.md>), [autonomous](<https://devfeed.tech/tags/autonomous.md>), [autonomous-driving](<https://devfeed.tech/tags/autonomous-driving.md>), [china](<https://devfeed.tech/tags/china.md>), [cities](<https://devfeed.tech/tags/cities.md>), [cost](<https://devfeed.tech/tags/cost.md>), [costs](<https://devfeed.tech/tags/costs.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [decoder](<https://devfeed.tech/tags/decoder.md>), [dependencies](<https://devfeed.tech/tags/dependencies.md>), [deployment](<https://devfeed.tech/tags/deployment.md>), [detection](<https://devfeed.tech/tags/detection.md>), [framework](<https://devfeed.tech/tags/framework.md>), [generate](<https://devfeed.tech/tags/generate.md>), [generation](<https://devfeed.tech/tags/generation.md>), [industrial](<https://devfeed.tech/tags/industrial.md>), [map](<https://devfeed.tech/tags/map.md>), [network](<https://devfeed.tech/tags/network.md>), [updating](<https://devfeed.tech/tags/updating.md>)

### AI overview

PGU-Decoder combines three road-map updating tasks--intra-intersection network generation, merging/diverging detection, and drivable-area detection--in one decoding framework. Its Prior Query Module samples road-network geometries to create point-level queries and keeps their reference points fixed during decoding; point representations are aggregated into instances, whose dependencies are modeled by a Relational Graph Attention Network before task-specific prediction heads produce outputs. The abstract reports evaluation across more than 360 Chinese cities, outperforming seven baselines. After deployment at AutoNavi Map, reported quality scores exceeded 98% on all three tasks, manual verification costs fell by 95%, and update frequency rose from weekly to hourly.

### Source excerpt

arXiv:2611.23936v1 Announce Type: new Abstract: Timely road-network map updates are essential to navigation and autonomous driving. Three key tasks are intra-intersection road network generation, merging/diverging detection, and drivable-area detection, covering topology, geometry, and attributes, respectively. Separate pipelines duplicate rule maintenance, quality assurance, and manual verification, increasing operational costs. We propose PGU-Decoder, a production-grade unified decoding framework for semantic prediction across these tasks. A Prior Query Module (PQM) constructs point-level prior queries from sampled locations on road-network geometries. Fixed reference points preserve positional priors throughout decoding, allowing the model to focus on surrounding context. Point-level representations are aggregated into instance-level representations, and a Relational Graph Attention Network (RGAT) captures inter-instance dependencies. Task-specific heads then generate predictions. We validate PGU-Decoder in large-scale automated road-network map updating across more than 360 cities in China, where it consistently outperforms seven baselines. Following deployment at AutoNavi Map, it achieves quality scores above 98% on all three tasks, reduces manual verification costs by 95%, and increases update frequency from weekly to hourly. These results demonstrate a practical and cost-effective industrial solution for road-network map updating.

## Improving Cross-Lingual Token Representations by Adding a Pinch of SALT

DevFeed: [Improving Cross-Lingual Token Representations by Adding a Pinch of SALT](<https://devfeed.tech/articles/improving-cross-lingual-token-representations-by-adding-a-pinch-of-salt-142559.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2609.09953>)

Author: Guillem Ram\\'irez

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Representation learning](<https://devfeed.tech/topics/representation-learning.md>), [Enhancing AI model performance](<https://devfeed.tech/topics/enhancing-ai-model-performance.md>), [Embeddings](<https://devfeed.tech/topics/embeddings.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [Internationalization (i18n)](<https://devfeed.tech/topics/i18n.md>)

Tags: [alignment](<https://devfeed.tech/tags/alignment.md>), [classification](<https://devfeed.tech/tags/classification.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [detection](<https://devfeed.tech/tags/detection.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [hallucination-detection](<https://devfeed.tech/tags/hallucination-detection.md>), [languages](<https://devfeed.tech/tags/languages.md>), [learning](<https://devfeed.tech/tags/learning.md>), [lightweight](<https://devfeed.tech/tags/lightweight.md>), [mining](<https://devfeed.tech/tags/mining.md>), [performance](<https://devfeed.tech/tags/performance.md>), [post-training](<https://devfeed.tech/tags/post-training.md>), [retrieval](<https://devfeed.tech/tags/retrieval.md>), [salt](<https://devfeed.tech/tags/salt.md>), [sequence](<https://devfeed.tech/tags/sequence.md>), [signal](<https://devfeed.tech/tags/signal.md>), [span](<https://devfeed.tech/tags/span.md>), [supervision](<https://devfeed.tech/tags/supervision.md>), [tasks](<https://devfeed.tech/tags/tasks.md>), [token](<https://devfeed.tech/tags/token.md>), [translation](<https://devfeed.tech/tags/translation.md>)

### AI overview

SALT improves token representations in cross-lingual sentence encoders by post-training them with span-level supervision. Across five multilingual token-level benchmarks, it achieves the best overall results on four, outperforming alternative fine-tuning strategies and competitive encoders. The abstract also reports gains on cross-lingual retrieval and classification, suggesting the method can benefit sentence-level performance too.

### Source excerpt

arXiv:2609.09953v2 Announce Type: replace-cross Abstract: Cross-lingual sentence encoders enable scalable transfer across hundreds of languages, powering applications such as translation mining and zero-shot learning in low-resource settings. Although trained for sentence-level alignment, they are increasingly also applied to token-level tasks such as hallucination detection and sequence tagging, exposing a mismatch between training and usage. We propose SALT, a lightweight post-training method that improves token representations by injecting span-level supervision into existing sentence encoders. Across five multilingual token-level benchmarks, SALT achieves the best overall results on four of them, outperforming alternative fine-tuning strategies and competitive encoders. It also improves sentence-level performance on cross-lingual retrieval and classification tasks. These results demonstrate that span-level supervision is an effective signal for improving both token and sentence representations.

## Cardiac MRI-informed electrocardiography enables biologically grounded cardiomyopathy detection

DevFeed: [Cardiac MRI-informed electrocardiography enables biologically grounded cardiomyopathy detection](<https://devfeed.tech/articles/cardiac-mri-informed-electrocardiography-enables-biologically-grounded-cardiomyopathy-detection-142725.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.23925>)

Author: Zhengyao Ding, Kewen Wang, Yujian Hu, Ziyu Li, Yiheng Mao, Yue Wang, Haitao Li, Dengnan Zhang, Xuanhong Liu, Yue Wang, Jing Wang, Yue Chen, Longbo Wang, Mengjia Chen, Chengchen Zhao, Xuesen Chu, Yubin

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Representation learning](<https://devfeed.tech/topics/representation-learning.md>), [Medical imaging](<https://devfeed.tech/topics/medical-imaging.md>), [AI Foundation Models](<https://devfeed.tech/topics/ai-foundation-models.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [accessibility](<https://devfeed.tech/tags/accessibility.md>), [assessment](<https://devfeed.tech/tags/assessment.md>), [biology](<https://devfeed.tech/tags/biology.md>), [classification](<https://devfeed.tech/tags/classification.md>), [correlation](<https://devfeed.tech/tags/correlation.md>), [cost](<https://devfeed.tech/tags/cost.md>), [cross](<https://devfeed.tech/tags/cross.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [detection](<https://devfeed.tech/tags/detection.md>), [ecg](<https://devfeed.tech/tags/ecg.md>), [external](<https://devfeed.tech/tags/external.md>), [foundation](<https://devfeed.tech/tags/foundation.md>), [foundation-models](<https://devfeed.tech/tags/foundation-models.md>), [framework](<https://devfeed.tech/tags/framework.md>), [functional](<https://devfeed.tech/tags/functional.md>), [genome](<https://devfeed.tech/tags/genome.md>), [grounded](<https://devfeed.tech/tags/grounded.md>), [imaging](<https://devfeed.tech/tags/imaging.md>), [independent](<https://devfeed.tech/tags/independent.md>), [q-bio-qm](<https://devfeed.tech/tags/q-bio-qm.md>)

### AI overview

MyoECG transfers structural and functional information from cardiac MRI (CMR) into standard 12-lead ECGs, aiming to detect cardiomyopathy and classify subtypes using ECGs alone. Pretrained on 45,241 paired ECG-CMR examinations, it was validated across more than 1.8 million ECGs from seven independent datasets. For cardiomyopathy detection, it outperformed five pretrained ECG models, with AUROCs of 0.800-0.974 across datasets with prevalence from 0.2% to 54.7%. In a study involving six cardiologists, assistance from MyoECG improved mean sensitivity and specificity. Its score also showed genetic and circulating-protein associations; the genetic correlation with cardiomyopathy became non-significant after adjustment for 27 measured CMR traits, consistent with the score capturing CMR-derived information. The abstract reports these findings but does not provide details on clinical implementation or prospective impact.

### Source excerpt

arXiv:2611.23925v1 Announce Type: new Abstract: Precise myocardial phenotyping at population scale is limited by the cost and accessibility of cardiac magnetic resonance (CMR), whereas electrocardiography (ECG) is widely available but reflects electrical rather than structural and functional information. Here we develop MyoECG, a cross-modal representation learning framework that distils myocardial structural and functional knowledge from CMR into standard 12-lead ECGs, enabling imaging-informed cardiomyopathy detection and subtype classification from ECGs alone. MyoECG was pretrained on 45,241 paired ECG-CMR examinations from UK Biobank and a tertiary cardiovascular centre, then developed and validated across more than 1.8 million ECGs from seven independent datasets. Across internal and external datasets spanning prevalences of 0.2-54.7%, MyoECG outperformed five pretrained ECG foundation models, with AUROCs of 0.800-0.974 for cardiomyopathy detection. In a multi-reader study of six cardiologists, MyoECG assistance improved mean sensitivity by 0.109 (95% CI, 0.041-0.175; P = 0.009) and specificity by 0.056 (95% CI, 0.004-0.108; P = 0.040). In 41,128 UK Biobank participants, the MyoECG score showed genetic correlations with cardiomyopathy (rg = 0.409) and dilated cardiomyopathy (rg = 0.371), genome-wide associations at four loci, including regions mapped to NFIA and KCNQ1, and associations with 69 circulating proteins related to myocardial stress, cell adhesion, vascular biology and metabolism. The cardiomyopathy correlation attenuated to non-significance after adjustment for 27 measured CMR traits (rg = 0.131; P = 0.143), indicating that the score captures CMR-derived information rather than dataset-specific shortcuts. MyoECG offers a scalable route to CMR-informed cardiomyopathy assessment and a strategy for transferring knowledge from resource-intensive imaging to widely deployable physiological signals.

## Training-free motion statistics match self-supervised behavior encoders

DevFeed: [Training-free motion statistics match self-supervised behavior encoders](<https://devfeed.tech/articles/training-free-motion-statistics-match-self-supervised-behavior-encoders-143562.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.24792>)

Author: Ziwen Chen

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Representation learning](<https://devfeed.tech/topics/representation-learning.md>), [Post-training optimization](<https://devfeed.tech/topics/post-training-optimization.md>)

Tags: [behavior](<https://devfeed.tech/tags/behavior.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [free](<https://devfeed.tech/tags/free.md>), [motion](<https://devfeed.tech/tags/motion.md>), [statistics](<https://devfeed.tech/tags/statistics.md>), [training](<https://devfeed.tech/tags/training.md>)

### AI overview

A training-free motion representation matched or outperformed the compared self-supervised encoders across the reported benchmarks. It summarizes generic per-frame measurements over fixed time windows, reduces them with PCA, and compares equal-size frozen representations using one shared classifier. It scored 85.8 macro F1 on CalMS21 versus 82.4 for the best encoder, tied using four labeled videos, and led in tests involving annotator changes and few-video learning. It also outscored released or published methods on MABe22 mouse triplets (83.7 vs. 80.3), fly groups (51.6 vs. 48.2 mean F1), and Shot7M2 (68.8 vs. 64.7); hBehaveMAE remained better on the shortest movements. Short windows accounted for much of the result. Raw body-frame keypoints alone matched or beat encoders on both mouse datasets and Shot7M2, but fell short on fly groups, where adding measurements improved results by 5.3 points. These findings are limited to the reported benchmarks.

### Source excerpt

arXiv:2611.24792v1 Announce Type: cross Abstract: Self-supervised encoders for animal behavior are often trained on, or fed, hand-built movement features. We ask whether they beat those ingredients when compared fairly. A training-free representation, made of generic per-frame measurements summarized over fixed time windows and reduced with PCA, is compared with retrained and released encoders (BAMS, hBehaveMAE) under a preregistered protocol: frozen representations of equal size and one shared classifier. On CalMS21 it reaches 85.8 macro F1 against 82.4 for the best encoder, ties with four labelled videos, and leads when the annotator changes or new behaviors must be learned from a few videos; the reading holds for encoders retrained with three seeds. On the MABe22 mouse triplets it scores 83.7 against 80.3 for the released hBehaveMAE (gap +3.4, 95% interval +2.1 to +4.7). On a second species, the MABe22 fly groups, the same recipe scores 51.6 mean F1 over 50 tasks against 48.2 for BAMS, the best published method. On Shot7M2, the synthetic basketball benchmark built by hBehaveMAE's authors, it scores 68.8 against 64.7 for hBehaveMAE, the best method in that paper, though hBehaveMAE remains better on the shortest movements. Short time windows carry most of the result. Raw keypoints in each body's own frame, summarized over the same windows, already match or beat the encoders on both mouse datasets and on Shot7M2; on the fly groups they fall short, and the measurements add 5.3 points. On these benchmarks, current behavior encoders have yet to beat the hand-built features they learn from, and such baselines should be reported alongside them.

[Next page](<https://devfeed.tech/topics/representation-learning.md?cursor=WyIyMDI2LTExLTE5VDA1OjAwOjAwKzAwOjAwIiwgIjljYjNjYTE1LTgyZDQtNDhmOC05NmY5LTJmODVkOTdlMzc5NSJd>)