# Self-supervised learning

A machine learning technique that generates implicit labels from unstructured data rather than relying on manually labeled datasets.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## RePAIR: Predictive Self-Supervised Representation Learning in Chess

DevFeed: [RePAIR: Predictive Self-Supervised Representation Learning in Chess](<https://devfeed.tech/articles/repair-predictive-self-supervised-representation-learning-in-chess-144905.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2606.11860>)

Author: Christoph Koller, Johannes F\\"urnkranz, Timo Bertram

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Representation learning](<https://devfeed.tech/topics/representation-learning.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Self-supervised learning](<https://devfeed.tech/topics/self-supervised-learning.md>), [Embeddings](<https://devfeed.tech/topics/embeddings.md>), [bert](<https://devfeed.tech/topics/bert.md>)

Tags: [autoencoders](<https://devfeed.tech/tags/autoencoders.md>), [bert](<https://devfeed.tech/tags/bert.md>), [chess](<https://devfeed.tech/tags/chess.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [learning](<https://devfeed.tech/tags/learning.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [representation-learning](<https://devfeed.tech/tags/representation-learning.md>), [sequence](<https://devfeed.tech/tags/sequence.md>), [space](<https://devfeed.tech/tags/space.md>)

### AI overview

RePAIR learns compact representations of consecutive chess positions by masking large portions of latent-state sequences, then using a lightweight predictor to repair the gaps in a lower-dimensional embedding space. The authors report that its encoder clusters meaningful chess concepts, and that reconstructing masked board states captures piece movements without reinforcement learning. They also use trajectories through the representation space to inspect games. These findings are reported for chess; the supplied excerpt does not establish performance on other sequential data.

### Source excerpt

arXiv:2606.11860v2 Announce Type: replace Abstract: In this paper, we introduce Representation Prediction via Autoencoding using Iterative Refinement (RePAIR) - a novel self-supervised representation learning architecture that synthesizes Masked Autoencoders (MAE), Joint Embedding Predictive Architectures (JEPA), and Bidirectional Encoder Representations from Transformers (BERT). We demonstrate how it can be used to encode objects in sequential data like consecutive chess positions into compact yet meaningful representations. The basic principle of the architecture is to mask large portions of a sequence of latent states, similar to BERT and MAE. Then, we apply a lightweight Predictor to the latent representations that repairs gaps in the sequence in a lower-dimensional embedding space akin to JEPA. Our experiments in the domain of chess show that the Encoder refines the board representations such that meaningful chess concepts emerge clustered in the latent space. Furthermore, reconstructions of the masked board states show that the model is able to reason about the piece movements without relying on costly reinforcement learning methods. Lastly, we find that the resulting representation space allows for quick and intuitive dissections of chess games by observing the game path trajectories in this semantically rich space.

## Predictive Self-Supervised Learning Provably Identifies Stochastic Signals under Nuisance

DevFeed: [Predictive Self-Supervised Learning Provably Identifies Stochastic Signals under Nuisance](<https://devfeed.tech/articles/predictive-self-supervised-learning-provably-identifies-stochastic-signals-under-nuisance-144945.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2609.37789>)

Author: Fabian A. Mikulasch, Friedemann Zenke

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Self-supervised learning](<https://devfeed.tech/topics/self-supervised-learning.md>), [Representation learning](<https://devfeed.tech/topics/representation-learning.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>)

Tags: [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [self-supervised-learning](<https://devfeed.tech/tags/self-supervised-learning.md>), [signal](<https://devfeed.tech/tags/signal.md>)

### AI overview

The paper proves that predictive self-supervised learning can identify a stochastic, prediction-relevant signal despite observation-private nuisance. It attributes this to two principles: predictive mutual information maximization preserves information needed for prediction, while latent distribution matching constrains its encoding enough to make the signal identifiable. In simulations, Gaussian predictors recover the true signal up to an affine transformation, including in dynamic environments with nuisance.

### Source excerpt

arXiv:2609.37789v2 Announce Type: replace Abstract: Self-supervised learning (SSL) by predicting in latent space, without generating the input data itself, learns highly abstract, useful representations. Intuitively, this success is often attributed to its ability to discard nuisance information that is irrelevant to prediction. However, this poses a conundrum: both stochastic variation in a prediction-relevant latent signal and true nuisance make observations partly unpredictable; how could they be distinguished? Surprisingly, we prove that common SSL methods can achieve exactly this, by implicitly instantiating a latent-variable model with stochastic dynamics and observation-private nuisance. We trace their ability to recover the stochastic signal to two complementary principles: Predictive mutual information maximization ensures that representations retain the information needed for prediction, while latent distribution matching constrains how this information is encoded, thereby making the retained signal identifiable. We confirm this identifiability result in simulations for Gaussian predictors, which recover the true signal up to an affine transformation even in dynamic, nuisance-laden environments.

## Implicit Neural Representations of Individual Behavior

DevFeed: [Implicit Neural Representations of Individual Behavior](<https://devfeed.tech/articles/implicit-neural-representations-of-individual-behavior-144906.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2606.12200>)

Author: Andrew Kang, Priya Narasimhan

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Self-supervised learning](<https://devfeed.tech/topics/self-supervised-learning.md>), [Representation learning](<https://devfeed.tech/topics/representation-learning.md>), [Neural Network](<https://devfeed.tech/topics/neural-network.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>)

Tags: [actions](<https://devfeed.tech/tags/actions.md>), [agents](<https://devfeed.tech/tags/agents.md>), [annotations](<https://devfeed.tech/tags/annotations.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [avoid](<https://devfeed.tech/tags/avoid.md>), [behavior](<https://devfeed.tech/tags/behavior.md>), [behaviors](<https://devfeed.tech/tags/behaviors.md>), [chess](<https://devfeed.tech/tags/chess.md>), [continuous](<https://devfeed.tech/tags/continuous.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [data](<https://devfeed.tech/tags/data.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [distribution](<https://devfeed.tech/tags/distribution.md>), [environments](<https://devfeed.tech/tags/environments.md>), [film](<https://devfeed.tech/tags/film.md>), [fixed](<https://devfeed.tech/tags/fixed.md>), [formula-1](<https://devfeed.tech/tags/formula-1.md>), [formula-1-racing](<https://devfeed.tech/tags/formula-1-racing.md>), [function](<https://devfeed.tech/tags/function.md>), [games](<https://devfeed.tech/tags/games.md>), [neural](<https://devfeed.tech/tags/neural.md>)

### AI overview

Behavioral INR infers policy identity from unlabeled episodes by representing each policy as a state-to-action function, modulated by an episode-level latent through FiLM layers. This self-supervised generative approach accommodates variable episode lengths and sampling granularities. The paper defines policy-level OOD shifts along state and action distribution axes and evaluates on synthetic data, MuJoCo, chess, Formula 1 racing, robotics, and Seek-Avoid datasets. The abstract reports its most consistent gains in difficult continuous state-action settings, especially with longer episodes, more policies, and OOD splits; amortized history encoders remain competitive when symbolic repetition or low-dimensional action statistics reveal identity.

### Source excerpt

arXiv:2606.12200v2 Announce Type: replace Abstract: We study policy representation learning from unlabeled multi-policy behavioral data. Each episode is generated by a fixed policy, but policy labels are unavailable. This setting appears in robotics play, demonstrations, games, racing, and other datasets where heterogeneous behaviors are mixed without annotations. We introduce Behavioral INR, a self-supervised generative model that adapts implicit neural representations (INRs) from vision to behavior. Instead of mapping coordinates to RGB values, Behavioral INR represents a policy as a state-action function mapping states to subsequent actions. An episode-level latent modulates this function through FiLM layers, yielding a generative prior over policies and allowing policy identity to be inferred without supervision. Because INRs treat each datapoint as samples from an underlying function, the same model naturally accommodates variable episode lengths and different sampling granularities, as in vision INRs with different image resolutions. We also define policy-level out-of-distribution (OOD) shifts along state-distribution and action-distribution axes, which arise when policies overlap in states or actions but are not captured by standard behavioral OOD settings based only on new agents or environments. We evaluate on synthetic Gaussian random field data, MuJoCo demonstrations with controlled OOD splits, and real-world chess, Formula 1 racing, robotics, and Seek-Avoid datasets. Behavioral INR most consistently improves policy identifiability in the hardest continuous state-action settings, especially when longer episodes, more policies, and OOD splits reduce the usefulness of marginal shortcuts; amortized history encoders remain competitive when policy identity can be recovered from symbolic repetition or low-dimensional action statistics.

## Omni-modal decomposition autoencoders learn full-stack wearable disentangled representations

DevFeed: [Omni-modal decomposition autoencoders learn full-stack wearable disentangled representations](<https://devfeed.tech/articles/omni-modal-decomposition-autoencoders-learn-full-stack-wearable-disentangled-representations-144917.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2608.07385>)

Author: Ioannis Ziogas, Ensieh Khazaei, Bilal Taha, Aamna Al Shehhi, Ahsan H. Khandoker, Leontios J. Hadjileontiadis, Dimitrios Hatzinakos

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Representation learning](<https://devfeed.tech/topics/representation-learning.md>), [VAE](<https://devfeed.tech/topics/vae.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Self-supervised learning](<https://devfeed.tech/topics/self-supervised-learning.md>)

Tags: [autoencoders](<https://devfeed.tech/tags/autoencoders.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [eess-as](<https://devfeed.tech/tags/eess-as.md>), [eess-sp](<https://devfeed.tech/tags/eess-sp.md>), [full-stack](<https://devfeed.tech/tags/full-stack.md>), [fusion](<https://devfeed.tech/tags/fusion.md>), [generation](<https://devfeed.tech/tags/generation.md>), [series](<https://devfeed.tech/tags/series.md>), [stat-ml](<https://devfeed.tech/tags/stat-ml.md>), [vae](<https://devfeed.tech/tags/vae.md>), [wearable](<https://devfeed.tech/tags/wearable.md>)

### AI overview

Omni-modal Variational Decomposition Autoencoders (OmniDecVAEs) use a shared encoder-decoder to learn representations across heterogeneous wearable time-series modalities. They extend DecVAEs with per-modality time-frequency latent subspaces, a multi-view self-supervised decomposition loss, and a component-wise conditional decoder. On HARWE, UCI HAR, and WESAD, the abstract reports up to 3.0 percentage points higher DCI disentanglement for unseen subjects. On HARWE, results improve subject-dependent activity and identity recognition by 1.1 and 5.8 points; on unseen subjects in multi-device datasets, they exceed supervised end-to-end models by 31.6 and 22.3 points. Reconstruction error is 34-74% lower than with multi-modal VAEs, and synthetic data improve unseen-subject affect recognition by 2 points. The model has 4.1 million parameters, with parameter count reported as invariant to modality count and 21 times fewer parameters than a multi-branch VAE.

### Source excerpt

arXiv:2608.07385v2 Announce Type: replace Abstract: Learning disentangled representations is key for general-purpose multi-modal wearable models, yet existing approaches rarely combine classification, disentanglement, fusion, and generation of heterogeneous multi-modal time series in a single model. We introduce Omni-modal Variational Decomposition Autoencoders (OmniDecVAEs), a framework that learns multi-purpose representations from arbitrarily many modalities with a single shared encoder-decoder. OmniDecVAEs extend DecVAEs by learning time-frequency latent subspaces per modality through a multi-view self-supervised decomposition loss and a component-wise conditional decoder. On the omni-modal human activity recognition (HAR) HARWE dataset (up to thirty channels) and the public UCI HAR and WESAD datasets, OmniDecVAEs learn the most disentangled representations for unseen subjects (DCI disentanglement up to 3.0 percentage points (pp) higher). On HARWE, they improve subject-dependent HAR and identity recognition by 1.1 pp and 5.8 pp, and outperform supervised end-to-end models by +31.6 pp and +22.3 pp on unseen subjects of multi-device datasets. Moreover, their generative capacity improves reconstruction mean squared error over multi-modal VAEs (34-74% improvement) and their synthetic data improve affect recognition on unseen subjects (2 pp). With 4.1M parameters, invariant to the number of modalities (21xfewer than a multi-branch VAE), OmniDecVAEs are scalable, single-model full-stack learners for wearable representations.

## SHAMISA: SHAped Modeling of Implicit Structural Associations for Self-supervised No-Reference Image Quality Assessment

DevFeed: [SHAMISA: SHAped Modeling of Implicit Structural Associations for Self-supervised No-Reference Image Quality Assessment](<https://devfeed.tech/articles/shamisa-shaped-modeling-of-implicit-structural-associations-for-self-supervised-no-reference-image-quality-assessment-142511.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2603.13669>)

Author: Mahdi Naseri, Zhou Wang

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Image](<https://devfeed.tech/topics/image.md>), [Self-supervised learning](<https://devfeed.tech/topics/self-supervised-learning.md>), [metric learning](<https://devfeed.tech/topics/metric-learning.md>)

Tags: [annotations](<https://devfeed.tech/tags/annotations.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [assessment](<https://devfeed.tech/tags/assessment.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [bottleneck](<https://devfeed.tech/tags/bottleneck.md>), [comparisons](<https://devfeed.tech/tags/comparisons.md>), [construction](<https://devfeed.tech/tags/construction.md>), [content](<https://devfeed.tech/tags/content.md>), [continuous](<https://devfeed.tech/tags/continuous.md>), [control](<https://devfeed.tech/tags/control.md>), [cross](<https://devfeed.tech/tags/cross.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [dataset](<https://devfeed.tech/tags/dataset.md>), [diagnostics](<https://devfeed.tech/tags/diagnostics.md>), [embedding](<https://devfeed.tech/tags/embedding.md>), [encoder](<https://devfeed.tech/tags/encoder.md>), [image](<https://devfeed.tech/tags/image.md>), [modeling](<https://devfeed.tech/tags/modeling.md>), [quality](<https://devfeed.tech/tags/quality.md>), [quality-assessment](<https://devfeed.tech/tags/quality-assessment.md>), [reference](<https://devfeed.tech/tags/reference.md>)

### AI overview

SHAMISA trains a no-reference image quality encoder from unlabeled distorted images, avoiding IQA opinion-score labels and contrastive loss during pre-training. Its compositional distortion engine varies one distortion factor at a time across continuous degradation parameters, allowing soft, content-sensitive relations: images with shared distortion patterns are brought together, while severity changes create structured shifts. Dual-source relation graphs combine known degradation profiles with learned structural affinities. The encoder is then frozen and paired with a linear regressor. The abstract reports strong performance on synthetic, authentic, and cross-dataset benchmarks, improved transfer in the reported cross-dataset comparisons, and favorable qualitative robustness diagnostics; it gives no specific scores or benchmark names.

### Source excerpt

arXiv:2603.13669v3 Announce Type: replace-cross Abstract: No-Reference Image Quality Assessment (NR-IQA) aims to estimate perceptual quality without access to a pristine reference. Learning an NR-IQA model faces a fundamental bottleneck because it requires many costly human perceptual labels. We propose SHAMISA, a non-contrastive self-supervised framework that learns from unlabeled distorted images through explicitly structured relational supervision. Unlike prior methods that impose rigid binary similarity constraints, SHAMISA shapes soft, controllable implicit structural associations that are both distortion-aware and content-sensitive. A key innovation is its compositional distortion engine, which generates an uncountable family of degradations from continuous parameter spaces and groups the resulting images so that exactly one distortion factor varies at a time. This construction enables fine-grained control over representational similarity during training. Images with shared distortion patterns are pulled together in the embedding space, while severity variations produce structured shifts. Dual-source relation graphs integrate known degradation profiles with emergent structural affinities throughout pre-training. The learned encoder is then frozen and paired with a linear regressor. Extensive experiments on synthetic, authentic, and cross-dataset NR-IQA benchmarks demonstrate strong overall performance, improved transfer in the reported cross-dataset comparisons, and favorable qualitative robustness diagnostics. Self-supervised pre-training requires no IQA opinion-score annotations or contrastive loss.

## JEPA as a Transferable Optimizer: Learning and Understanding Black-Box Optimization from Reward-Agnostic Pretraining

DevFeed: [JEPA as a Transferable Optimizer: Learning and Understanding Black-Box Optimization from Reward-Agnostic Pretraining](<https://devfeed.tech/articles/jepa-as-a-transferable-optimizer-learning-and-understanding-black-box-optimization-from-reward-agnostic-pretraining-142678.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.23608>)

Author: Mayukh Debnath, Manan Roy Choudhury

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [model architecture](<https://devfeed.tech/topics/model-architecture.md>), [Self-supervised learning](<https://devfeed.tech/topics/self-supervised-learning.md>), [few-shot](<https://devfeed.tech/topics/few-shot.md>)

Tags: [architecture](<https://devfeed.tech/tags/architecture.md>), [bayesian](<https://devfeed.tech/tags/bayesian.md>), [black-box](<https://devfeed.tech/tags/black-box.md>), [control](<https://devfeed.tech/tags/control.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [gradient](<https://devfeed.tech/tags/gradient.md>), [history](<https://devfeed.tech/tags/history.md>), [information](<https://devfeed.tech/tags/information.md>), [learning](<https://devfeed.tech/tags/learning.md>), [llm](<https://devfeed.tech/tags/llm.md>), [memory](<https://devfeed.tech/tags/memory.md>), [model](<https://devfeed.tech/tags/model.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [policy](<https://devfeed.tech/tags/policy.md>), [prediction](<https://devfeed.tech/tags/prediction.md>), [pretraining](<https://devfeed.tech/tags/pretraining.md>), [random](<https://devfeed.tech/tags/random.md>), [reward](<https://devfeed.tech/tags/reward.md>)

### AI overview

JEPA-PS learns to propose black-box optimization candidates from short evaluation histories, predicting the next point in a trajectory rather than rewards or gradients. Pretrained on random rollouts with parameter vectors and scalar returns, it proposes candidates in one forward pass without task descriptions, reward shaping, or test-time gradients. Across 15 control environments and 12 optimizers, it was competitive with tuned classical baselines, transferred zero-shot to environments excluded from pretraining, and beat an LLM optimizer by more than 2x on Swimmer at 71x lower per-step cost. Its useful history varied by landscape: smooth tasks benefited from the full trajectory, while on rugged tasks the best-seen point carried nearly all the signal. Evidence that this pattern relates to fitness-distance correlation is weak and was not corroborated by a separate reward-ordering probe. Decoder ablations also found every trained decoder variant worse than the default frozen representation.

### Source excerpt

arXiv:2611.23608v1 Announce Type: cross Abstract: Black-box optimization evaluates candidates through simulations, rollouts, or experiments, often without gradients. Learned optimizers are usually judged by whether they outperform existing methods. We ask a complementary question: what information does a learned optimizer use from its search history? We introduce JEPA-PS (JEPA Policy Search), based on a Joint-Embedding Predictive Architecture. It is pretrained on random rollouts using parameter vectors and scalar returns. Its latent-prediction objective predicts the next point in an optimization trajectory rather than its reward or gradient, allowing us to examine what was tried and how well it performed. At test time, JEPA-PS reads a short evaluation history and proposes a candidate in one forward pass, without task descriptions, reward shaping, or test-time gradients. We evaluate it on 15 control environments against 12 optimizers, including evolutionary methods, Bayesian optimization, CMA-ES, and an LLM optimizer. JEPA-PS is competitive with tuned classical baselines and outperforms the LLM optimizer by more than 2x on Swimmer at 71x lower per-step cost. It also transfers zero-shot to environments excluded from pretraining. Null-model comparisons and decoder-training ablations indicate that its latent representation carries information beyond simple history statistics; every trained decoder variant performs worse than the default frozen representation. Our main finding is that the history JEPA-PS uses depends on the task. On smooth landscapes it uses the full trajectory, whereas on rugged landscapes the best-seen point carries nearly all the signal. This memory requirement has a weak positive correlation with fitness-distance correlation (FDC; r=0.112, n=7), while an independent reward-ordering sensitivity probe does not corroborate the relationship (r=-0.024, n=7). Code: https://github.com/mdebnath22/jepa-props.

## Semi-Supervised GAN-Based Segmentation of CTPA Scans for Pulmonary Emboli Boundary Extraction

DevFeed: [Semi-Supervised GAN-Based Segmentation of CTPA Scans for Pulmonary Emboli Boundary Extraction](<https://devfeed.tech/articles/semi-supervised-gan-based-segmentation-of-ctpa-scans-for-pulmonary-emboli-boundary-extraction-143473.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.24018>)

Author: Maria Thoma, Michalis Savelonas, Foteini Malli, Dimitris Iakovidis

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Image](<https://devfeed.tech/topics/image.md>), [Self-supervised learning](<https://devfeed.tech/topics/self-supervised-learning.md>)

Tags: [adversarial](<https://devfeed.tech/tags/adversarial.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [boundary](<https://devfeed.tech/tags/boundary.md>), [challenges](<https://devfeed.tech/tags/challenges.md>), [cross-entropy](<https://devfeed.tech/tags/cross-entropy.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [dataset](<https://devfeed.tech/tags/dataset.md>), [detection](<https://devfeed.tech/tags/detection.md>), [diagnosis](<https://devfeed.tech/tags/diagnosis.md>), [disease](<https://devfeed.tech/tags/disease.md>), [entropy](<https://devfeed.tech/tags/entropy.md>), [error](<https://devfeed.tech/tags/error.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [experimental](<https://devfeed.tech/tags/experimental.md>), [feature](<https://devfeed.tech/tags/feature.md>), [gans](<https://devfeed.tech/tags/gans.md>), [generative](<https://devfeed.tech/tags/generative.md>), [generator](<https://devfeed.tech/tags/generator.md>), [image](<https://devfeed.tech/tags/image.md>), [large](<https://devfeed.tech/tags/large.md>), [semi](<https://devfeed.tech/tags/semi.md>)

### AI overview

The proposed semi-supervised GAN segments pulmonary emboli in CTPA scans by using unlabeled scans alongside a limited labeled set. Dice and cross-entropy losses address class imbalance; feature matching encourages anatomically plausible segmentations, while self-training reinforces learning from high-confidence pseudo-labels. On a public dataset, the method reported a Dice coefficient of 88.75% and HD95 of 1.74, outperforming fully supervised baselines. These results are reported for the evaluated dataset.

### Source excerpt

arXiv:2611.24018v1 Announce Type: cross Abstract: Pulmonary embolism (PE) is a pulmonary vascular disease associated with high morbidity and mortality, where delayed diagnosis can be fatal. While computed tomography pulmonary angiography (CTPA) is the clinical gold standard for PE detection, interpreting these scans manually is time-consuming and error-prone. In this work, we propose a semi-supervised segmentation method, employing generative adversarial networks (GANs) to leverage large pools of unlabeled CTPA scans. The proposed method addresses more effectively the challenges of CTPA image segmentation than previous methods by employing the dice loss and the cross-entropy loss to tackle the highly imbalanced nature of CTPA scans, the feature matching loss to encourage the generator to produce anatomically plausible segmentation maps that are indistinguishable from ground truth, as well as the self-training loss to reinforce learning from highly-confident pseudo-labeled samples. The experimental evaluation on a publicly available dataset shows that the proposed method maintains accurate segmentation, despite a limited number of labeled CTPA scans. with a Dice coefficient of 88.75% and a 95th percentile Hausdorff distance (HD95) of 1.74, outperforming fully supervised baselines.

## UCLIR: Unsupervised Continual Learning using Independent Representations

DevFeed: [UCLIR: Unsupervised Continual Learning using Independent Representations](<https://devfeed.tech/articles/uclir-unsupervised-continual-learning-using-independent-representations-143483.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.24093>)

Author: Kamel Aouaidjia, Yanchao Duan, Zihui Zhen, Xin Li, Yongfei Lv, Chongsheng Zhang

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Representation learning](<https://devfeed.tech/topics/representation-learning.md>), [Neural Network](<https://devfeed.tech/topics/neural-network.md>), [Self-supervised learning](<https://devfeed.tech/topics/self-supervised-learning.md>)

Tags: [autoencoders](<https://devfeed.tech/tags/autoencoders.md>), [cifar-10](<https://devfeed.tech/tags/cifar-10.md>), [continual-learning](<https://devfeed.tech/tags/continual-learning.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [independent](<https://devfeed.tech/tags/independent.md>), [linear](<https://devfeed.tech/tags/linear.md>), [mnist](<https://devfeed.tech/tags/mnist.md>), [tiny](<https://devfeed.tech/tags/tiny.md>)

### AI overview

UCLIR aims to reduce forgetting in continual learning by regularizing several small, independently learned complementary representations instead of one large shared representation. Independent autoencoders learn different input components through unsupervised regularization, so updates for new tasks are localized to particular representations and may interfere less with earlier knowledge. A linear classifier is used at inference for evaluation. On task sequences built from 5-Splits MNIST, five datasets, 5-Splits CIFAR-10, and 10-Splits Tiny ImageNet, the authors report results competitive with supervised and replay-based methods. The supplied text does not provide quantitative comparisons.

### Source excerpt

arXiv:2611.24093v1 Announce Type: new Abstract: In continual learning, regularization-based methods aim to mimic neuroplasticity by updating network weights to learn new tasks while constraining changes to parameters that are important for previously learned tasks through penalty terms. However, as the number of tasks increases, preserving knowledge from earlier tasks becomes less effective. This is mainly because updates for new tasks are distributed across the entire network, where parameters are highly interdependent due to the dense connectivity between neurons. Inspired by the Exponential Distance Rule (EDR), which states that the number of neural connections decreases with increasing physical distance, learning is naturally locally correlated while inter-area updates remain less task-dependent. Motivated by this principle, in this paper we demonstrate that applying weight regularization to multiple small and independently learned complementary representations is more effective than regularizing a single large shared representation. By decomposing the input into independently learned components, learning new tasks induces only localized parameter updates within specific representation, which limit interference and improve knowledge retention across tasks. We employ independent autoencoders that learn different parts of the input independently through unsupervised regularization. At inference time, a linear classifier is used for evaluation. The experimental results on a sequence of tasks using 5-Splits MNIST, sequence of 5-datasets, 5-Splits CIFAR-10, 10-Splits Tiny ImageNet shows competitive results with supervised and replay-based methods. Code is available.

## Uncovering the Invisible in Routine Radiology: Cross-Modal Detection of MRI-Verified Soft-Tissue Pathologies from CBCT via Physics-Guided 3D Self-Supervised Learning

DevFeed: [Uncovering the Invisible in Routine Radiology: Cross-Modal Detection of MRI-Verified Soft-Tissue Pathologies from CBCT via Physics-Guided 3D Self-Supervised Learning](<https://devfeed.tech/articles/uncovering-the-invisible-in-routine-radiology-cross-modal-detection-of-mri-verified-soft-tissue-pathologies-from-cbct-via-physics-guided-3d-self-supervised-learning-141834.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.23042>)

Author: Evgeny Alves Limarenko

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Medical imaging](<https://devfeed.tech/topics/medical-imaging.md>), [Physics-guided deep learning](<https://devfeed.tech/topics/physics-guided-deep-learning.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Self-supervised learning](<https://devfeed.tech/topics/self-supervised-learning.md>), [Representation learning](<https://devfeed.tech/topics/representation-learning.md>)

Tags: [3d](<https://devfeed.tech/tags/3d.md>), [architectures](<https://devfeed.tech/tags/architectures.md>), [autoencoder](<https://devfeed.tech/tags/autoencoder.md>), [automated](<https://devfeed.tech/tags/automated.md>), [beam](<https://devfeed.tech/tags/beam.md>), [consistency](<https://devfeed.tech/tags/consistency.md>), [cross](<https://devfeed.tech/tags/cross.md>), [cross-validation](<https://devfeed.tech/tags/cross-validation.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [data](<https://devfeed.tech/tags/data.md>), [deep-learning](<https://devfeed.tech/tags/deep-learning.md>), [detection](<https://devfeed.tech/tags/detection.md>), [disease](<https://devfeed.tech/tags/disease.md>), [eess-iv](<https://devfeed.tech/tags/eess-iv.md>), [flow](<https://devfeed.tech/tags/flow.md>), [framework](<https://devfeed.tech/tags/framework.md>), [gradient](<https://devfeed.tech/tags/gradient.md>), [learning](<https://devfeed.tech/tags/learning.md>), [modal](<https://devfeed.tech/tags/modal.md>), [physics](<https://devfeed.tech/tags/physics.md>), [routine](<https://devfeed.tech/tags/routine.md>), [self-supervised-learning](<https://devfeed.tech/tags/self-supervised-learning.md>)

### AI overview

In 212 paired TMJ CBCT-MRI joints, a physics-guided 3D learning pipeline detected a sub-visual signal for MRI-verified synovial effusion (ROC AUC 0.793). It used a frozen masked autoencoder pre-trained on 6,550 unlabeled CBCT volumes, with PhysLoss and the GCond optimizer directing learning toward the −300 to 300 HU soft-tissue window, followed by automated localization and cropping that removed 81% of background osseous variance. Under patient-isolated cross-validation, only representations trained with PhysLoss and GCond captured anatomical chirality and intra-patient consistency; the same architectures using standard L2 loss or MVC performed at random-guessing levels. The findings are limited to this TMJ study and support potential opportunistic screening from routine CBCT, not replacement of MRI.

### Source excerpt

arXiv:2611.23042v1 Announce Type: cross Abstract: Routine X-ray modalities like cone-beam computed tomography (CBCT) are optimized for high-contrast mineralized tissues, leaving soft-tissue abnormalities buried beneath photon noise and below human visual perception. Thus, detecting non-calcified pathologies requires secondary magnetic resonance imaging (MRI). We demonstrate that sub-visual radiographic signatures of soft-tissue pathology are physically preserved within routine CBCT scans and can be extracted for opportunistic screening using a targeted, physics-guided deep learning pipeline, even under extreme paired-data scarcity. Using the temporomandibular joint (TMJ) as a model system (N = 212 paired CBCT-MRI joints), we address the severe overfitting of high-capacity 3D networks through a two-stage framework. First, we employ a frozen full-grid 3D masked autoencoder (BertsWin) pre-trained on 6,550 unlabeled CBCT volumes via the GradientConductor (GCond) optimizer and an anatomy-aware objective (PhysLoss) redirecting gradient flow into the [-300, 300] HU soft-tissue window. Second, automated localization of the condylar zenith drives a deterministic 3D cropping that eliminates 81% of background osseous variance. Evaluated via a patient-isolated 3x10-fold stratified cross-validation protocol, a statistically robust sub-visual signal for MRI-verified synovial effusion (ROC AUC = 0.793) emerges exclusively from representations pre-trained with the PhysLoss and GCond regime. Crucially, only this setup captures true 3D anatomical chirality (p < 0.001) and high intra-patient structural consistency (0.791) over inter-patient noise; conversely, identical architectures pre-trained with standard L2 loss or unmasked variance decomposition (MVC) collapse to random guessing. These findings establish a generalizable paradigm for repurposing routine, low-contrast radiographic scans into opportunistic screening tools for occult soft-tissue disease.

## RelPrism: Multi-Faceted Self-Supervised Learning on Relational Databases

DevFeed: [RelPrism: Multi-Faceted Self-Supervised Learning on Relational Databases](<https://devfeed.tech/articles/relprism-multi-faceted-self-supervised-learning-on-relational-databases-141567.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2605.23241>)

Author: Jinyu Yang, Cheng Yang, Junze Chen, Zedi Liu, Muhan Zhang, Hanyang Peng, Chuan Shi

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Self-supervised learning](<https://devfeed.tech/topics/self-supervised-learning.md>), [Representation learning](<https://devfeed.tech/topics/representation-learning.md>), [Predictive AI](<https://devfeed.tech/topics/predictive-ai.md>)

Tags: [classification](<https://devfeed.tech/tags/classification.md>), [clustering](<https://devfeed.tech/tags/clustering.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [databases](<https://devfeed.tech/tags/databases.md>), [regression](<https://devfeed.tech/tags/regression.md>), [relational-databases](<https://devfeed.tech/tags/relational-databases.md>), [self-supervised-learning](<https://devfeed.tech/tags/self-supervised-learning.md>)

### AI overview

RelPrism addresses the limited adaptation support of self-supervised pretraining that draws supervision from only one facet of a relational database. It constructs intrinsic, relational, and hybrid attributes, then uses multi-granularity clustering of each perspective to create pseudo-task pools for pretraining. The authors report results across 14 tasks and 5 real-world datasets: ROC-AUC improves by 4.15% on classification tasks and MAE decreases by 10.75% on regression tasks versus state-of-the-art baselines. These are reported aggregate results; the supplied abstract does not detail individual tasks or evaluation conditions.

### Source excerpt

arXiv:2605.23241v2 Announce Type: replace Abstract: Relational databases (RDBs) remain the backbone of modern data systems and support a broad spectrum of predictive tasks that require multi-faceted information from different perspectives and granularities. For example, user churn prediction is often formulated as a binary classification task that relies more on user interaction patterns reflecting activity levels, whereas consumption value estimation is a fine-grained regression task that may require composite evidence from both user attributes and user--item interaction behaviors. Therefore, a central challenge is to learn representations with comprehensive coverage of the information underlying RDB tasks. A natural solution is to leverage self-supervised learning (SSL) for pre-training. However, existing SSL methods typically derive supervision from a single facet, such as node-level intrinsic attributes or subgraph-level relational structures. Consequently, their pre-training objectives provide limited support for adaptation. To this end, we propose RelPrism, a multi-faceted self-supervised learning framework for RDBs. Concretely, RelPrism first constructs intrinsic, relational, and hybrid attributes from distinct perspectives. It then applies multi-granularity clustering to each attribute perspective to form corresponding pseudo-task pools. Pre-training over these task pools exposes representations to comprehensive attribute perspectives and granularity levels, providing a stronger basis for downstream adaptation. Extensive experiments on 14 tasks across 5 real-world datasets show that RelPrism improves ROC-AUC by 4.15% on classification tasks and reduces MAE by 10.75% on regression tasks over state-of-the-art baselines. Our code is available at https://github.com/BUPT-GAMMA/RelPrism.

## Urban Representation Learning for Identifying Potential Disparities in Crime Reporting

DevFeed: [Urban Representation Learning for Identifying Potential Disparities in Crime Reporting](<https://devfeed.tech/articles/urban-representation-learning-for-identifying-potential-disparities-in-crime-reporting-141679.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.21825>)

Author: Jorge Valverde-Rebaza, Carlos Paez De La Cruz, Pedro P\\'erez-Collado, Daniel S\\'anchez-Cata\\~no, Eashley Mart\\'inez-Vergara

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Representation learning](<https://devfeed.tech/topics/representation-learning.md>), [Self-supervised learning](<https://devfeed.tech/topics/self-supervised-learning.md>), [outlier-detection](<https://devfeed.tech/topics/outlier-detection.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>)

Tags: [analysis](<https://devfeed.tech/tags/analysis.md>), [auditing](<https://devfeed.tech/tags/auditing.md>), [crime](<https://devfeed.tech/tags/crime.md>), [cs-cy](<https://devfeed.tech/tags/cs-cy.md>), [cs-ir](<https://devfeed.tech/tags/cs-ir.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [reporting](<https://devfeed.tech/tags/reporting.md>), [representation-learning](<https://devfeed.tech/tags/representation-learning.md>)

### AI overview

The framework flags regions whose crime-reporting profiles differ from those of representation-similar peers, providing an internal audit and prioritization signal rather than proof of underreporting. It builds hourly, weekly, and static profiles from georeferenced records, learns geographic self-supervised representations, and uses Neighbor Profile Divergence (NPD) to measure peer disagreement. In more than 1.8 million Mexico City crime records across 1,058 H3 regions, its mean Silhouette score was 0.4052 versus 0.2863 for Hex2Vec. NPD rankings were stable across neighborhood sizes (Spearman correlations 0.9720-0.9951), and retrieved peers often extended beyond nearby areas. High-divergence regions showed spatial organization and, in some cases, incident-volume differences from peer expectations; these patterns warrant further investigation, not conclusions about missing crime.

### Source excerpt

arXiv:2611.21825v1 Announce Type: new Abstract: Open crime data support urban safety analysis but reflect reported incidents rather than the complete distribution of criminal activity, potentially introducing spatial distortions in downstream analysis. We propose a self-supervised urban representation learning framework for identifying unusual reporting patterns relative to representation-similar peers using only georeferenced crime records. The method constructs complementary hourly, weekly, and static reporting profiles, learns scale-specific representations under geographic self-supervision, and integrates them into a unified latent space. Neighbor Profile Divergence (NPD) is then used to quantify disagreement between each region's reporting profile and those of its retrieved peers. Experiments on more than 1.8 million crime records from Mexico City, aggregated into 1,058 H3 regions, show that the proposed representation achieves a mean Silhouette score of 0.4052, compared with 0.2863 for Hex2Vec. NPD rankings remain highly stable across neighborhood sizes, with Spearman correlations from 0.9720 to 0.9951, while most high-divergence candidates persist across configurations. The retrieved peers also extend substantially beyond immediate geographic neighborhoods, indicating that the learned representation captures structure beyond local spatial proximity. High-divergence regions exhibit spatially organized patterns and, in several cases, differences in incident volume relative to peer expectations. These results support NPD as an internal auditing and prioritization mechanism for identifying regions that warrant further investigation, rather than as direct evidence of underreporting.

## Centroid Spectral Coupling: Implicit Alignment and Representation Diversity in Self-Supervised Learning

DevFeed: [Centroid Spectral Coupling: Implicit Alignment and Representation Diversity in Self-Supervised Learning](<https://devfeed.tech/articles/centroid-spectral-coupling-implicit-alignment-and-representation-diversity-in-self-supervised-learning-141742.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.22158>)

Author: M. Hadi Sepanj, Benyamin Ghojogh, Saed Moradi, Paul Fieguth

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Self-supervised learning](<https://devfeed.tech/topics/self-supervised-learning.md>), [Representation learning](<https://devfeed.tech/topics/representation-learning.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>)

Tags: [alignment](<https://devfeed.tech/tags/alignment.md>), [computer-vision](<https://devfeed.tech/tags/computer-vision.md>), [coupling](<https://devfeed.tech/tags/coupling.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [diversity](<https://devfeed.tech/tags/diversity.md>), [self-supervised-learning](<https://devfeed.tech/tags/self-supervised-learning.md>), [stat-ml](<https://devfeed.tech/tags/stat-ml.md>)

### AI overview

Centroid Spectral Coupling (CSC) combines view alignment, representation diversity, and collapse resistance in one self-supervised learning objective. It forms centroids from independently normalized augmented-view pairs and applies a ridge-based spectral objective to their uncentered second-moment matrix. The authors report an exact decomposition into nonnegative alignment and spectral-imbalance components, finite-loss guarantees for alignment and diversity, and complete constant-output collapse as a strict saddle in normalized representation space. On CIFAR-10, CIFAR-100, and Imagenette, CSC achieved the best linear Top-1 accuracy among evaluated non-contrastive methods; it also led in kNN Top-1 accuracy on both CIFAR datasets. Increasing ridge parameter κ strengthened alignment but reduced effective rank, showing an empirical alignment-diversity trade-off.

### Source excerpt

arXiv:2611.22158v1 Announce Type: cross Abstract: Self-Supervised Learning (SSL) aims to align augmented views while preserving representation diversity and preventing collapse. Existing non-contrastive methods often address these goals using multiple weighted loss terms. We introduce Centroid Spectral Coupling (CSC), a single-term SSL objective that implicitly couples view alignment, representation diversity, and collapse resistance. CSC constructs centroids from independently normalized augmented-view pairs and applies a ridge-based spectral objective to their uncentered second-moment matrix. Preserving centroid magnitudes encodes view agreement while encouraging balanced representation energy across spectral directions. We establish an exact decomposition into nonnegative alignment and spectral-imbalance components, characterize the geometry of global optima, and derive finite-loss guarantees for alignment and diversity. We also show that complete constant-output collapse is a strict saddle in normalized representation space. Experiments on CIFAR-10, CIFAR-100, and Imagenette show that CSC achieves the highest linear Top-1 accuracy among the evaluated non-contrastive methods on all three benchmarks, and the highest kNN Top-1 accuracy on both CIFAR datasets. A ridge-parameter sensitivity study reveals an empirical alignment--diversity trade-off: increasing $\kappa$ strengthens view alignment but reduces representation effective rank. These results show that a unified spectral objective yields competitive visual representations without explicit contrastive negatives or separately weighted alignment and diversity penalties.

## SLICES: A Benchmark and Evaluation Framework for Self-Supervised Learning on Sparse ICU Time Series

DevFeed: [SLICES: A Benchmark and Evaluation Framework for Self-Supervised Learning on Sparse ICU Time Series](<https://devfeed.tech/articles/slices-a-benchmark-and-evaluation-framework-for-self-supervised-learning-on-sparse-icu-time-series-138799.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.18868>)

Author: Hannes Ill, Dimitrios Proios, Leo Anthony Celi, Peter J. Sch\\"uffler

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Self-supervised learning](<https://devfeed.tech/topics/self-supervised-learning.md>), [Benchmark](<https://devfeed.tech/topics/benchmark.md>), [Time Series](<https://devfeed.tech/topics/time-series.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [reproducibility](<https://devfeed.tech/topics/reproducibility.md>)

Tags: [architecture](<https://devfeed.tech/tags/architecture.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [classification](<https://devfeed.tech/tags/classification.md>), [code](<https://devfeed.tech/tags/code.md>), [comparisons](<https://devfeed.tech/tags/comparisons.md>), [contrastive-learning](<https://devfeed.tech/tags/contrastive-learning.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [data](<https://devfeed.tech/tags/data.md>), [data-processing](<https://devfeed.tech/tags/data-processing.md>), [embedding](<https://devfeed.tech/tags/embedding.md>), [error](<https://devfeed.tech/tags/error.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [framework](<https://devfeed.tech/tags/framework.md>), [hospital](<https://devfeed.tech/tags/hospital.md>), [metric](<https://devfeed.tech/tags/metric.md>), [objectives](<https://devfeed.tech/tags/objectives.md>), [prediction](<https://devfeed.tech/tags/prediction.md>), [self-supervised-learning](<https://devfeed.tech/tags/self-supervised-learning.md>), [slices](<https://devfeed.tech/tags/slices.md>), [time-series](<https://devfeed.tech/tags/time-series.md>)

### AI overview

SLICES standardizes comparisons of self-supervised learning for sparse ICU time series by fixing 24-hour value and observation-mask inputs, a Transformer encoder, patient-level splits, four endpoints, two adaptation protocols, and five matched seeds across MIMIC-IV, eICU-CRD, and their pooled union. Of the tested SSL objectives, masked autoencoding (MAE) leads mean AUPRC in all nine frozen classification comparisons and seven of nine fine-tuning comparisons. Fine-tuned MAE leads hospital mortality at full labels and beats a same-architecture supervised Transformer at every label fraction; SSL fine-tuning also has the lowest length-of-stay error. Supervised baselines remain strongest for 24-hour mortality (GRU-D) and acute kidney injury (XGBoost). Rankings vary with adaptation protocol, comparator set, and probe-loss alignment. A two-seed MIMIC-IV rerun stays within two across-seed standard deviations in all 48 downstream runs. Generalization to unseen hospitals was not tested.

### Source excerpt

arXiv:2611.18868v1 Announce Type: new Abstract: Self-supervised learning (SSL) results for intensive care unit (ICU) time series are hard to compare when data processing, encoders, and adaptation vary together. We present SLICES, a benchmark and evaluation framework for comparing and extending SSL methods under a shared clinical setup. It fixes 24-hour value and observation-mask inputs, a Transformer encoder, patient-level splits, four endpoints, two adaptation protocols, and five matched seeds across MIMIC-IV, eICU-CRD, and their pooled union. Masked autoencoding (MAE), joint-embedding prediction (JEPA), and contrastive learning provide reference implementations. Among these objectives, MAE leads mean AUPRC in all nine frozen classification comparisons, also under converged probes, and in seven of nine fine-tuning comparisons. Fine-tuned MAE leads hospital mortality at full labels and scores higher than a supervised Transformer of the same architecture at every label fraction, and SSL fine-tuning gives the lowest length-of-stay error. Supervised baselines remain strong elsewhere: GRU-D leads 24-hour mortality and XGBoost leads acute kidney injury in all three regimes. Rankings depend on the adaptation protocol, the comparator set, and whether the probe loss matches the metric. A two-seed MIMIC-IV rerun with the maintained code stays within two across-seed SDs of the reference in all 48 downstream runs. The code repository supports new objectives, shared downstream evaluation, and regeneration of the reference results, so new SSL methods can be compared with these references under the same conditions. Generalization to unseen hospitals is not tested here and is the natural next extension.

## HeteroWear: Sensor-Aware Self-Supervised Learning for Heterogeneous Wearable Activity Recognition

DevFeed: [HeteroWear: Sensor-Aware Self-Supervised Learning for Heterogeneous Wearable Activity Recognition](<https://devfeed.tech/articles/heterowear-sensor-aware-self-supervised-learning-for-heterogeneous-wearable-activity-recognition-137378.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.19027>)

Author: Yasaman Mohammadpour, Pouya Shaeri

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Self-supervised learning](<https://devfeed.tech/topics/self-supervised-learning.md>), [Representation learning](<https://devfeed.tech/topics/representation-learning.md>), [Transformers](<https://devfeed.tech/topics/transformers.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [accelerometer](<https://devfeed.tech/tags/accelerometer.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [attention](<https://devfeed.tech/tags/attention.md>), [axis](<https://devfeed.tech/tags/axis.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [body](<https://devfeed.tech/tags/body.md>), [channel](<https://devfeed.tech/tags/channel.md>), [combines](<https://devfeed.tech/tags/combines.md>), [configuration](<https://devfeed.tech/tags/configuration.md>), [consistency](<https://devfeed.tech/tags/consistency.md>), [context](<https://devfeed.tech/tags/context.md>), [contribute](<https://devfeed.tech/tags/contribute.md>), [cross](<https://devfeed.tech/tags/cross.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [eess-sp](<https://devfeed.tech/tags/eess-sp.md>), [learning](<https://devfeed.tech/tags/learning.md>), [recognition](<https://devfeed.tech/tags/recognition.md>), [self-supervised-learning](<https://devfeed.tech/tags/self-supervised-learning.md>), [sensor](<https://devfeed.tech/tags/sensor.md>), [stat-ml](<https://devfeed.tech/tags/stat-ml.md>), [wearable](<https://devfeed.tech/tags/wearable.md>)

### AI overview

HeteroWear uses sensor-context embeddings, separate temporal and cross-sensor Transformers, and latent cross-attention to learn from wearable datasets with varied devices, body locations, sampling rates, and signal types. Pretraining combines structured sensor dropout, cross-configuration consistency, and per-axis masked sensor modeling; missing sensors are omitted rather than reconstructed. In leave-one-dataset-out pretraining and subject-disjoint tests across five benchmarks, it raised mean macro-F1 from 0.688 to 0.724 over the same architecture trained from scratch, with the largest gain on HHAR (0.823 vs. 0.741). It led single-accelerometer supervised baselines on four of five benchmarks, and held-out-device evaluation showed a mean +0.066 macro-F1 improvement. With 50% of test-time channels removed, it retained 73.8% of full-channel performance, versus 64.1% for the scratch-trained model without sensor dropout. Ablations attribute robustness mainly to structured sensor dropout, while consistency training and sensor-context encoding contribute to transfer.

### Source excerpt

arXiv:2611.19027v1 Announce Type: cross Abstract: Self-supervised learning for wearable sensing is usually confined to a single dataset or device family, leaving open whether representations transfer across heterogeneous sensor configurations and whether gains come from pretraining rather than architecture or richer sensor input. We present HeteroWear, a sensor-aware self-supervised framework pretrained on an eight-dataset wearable corpus (173 subjects, 358,261 windows) spanning inertial and physiological signals across body locations, devices, and sampling rates. HeteroWear separates within-sensor temporal modeling from cross-sensor relational modeling using sensor-context embeddings, temporal and cross-sensor Transformers, and latent cross-attention fusion. Training combines structured sensor dropout, cross-configuration consistency, and per-axis masked sensor modeling; missing sensors are omitted rather than reconstructed. Under leave-one-dataset-out pretraining and subject-disjoint evaluation on PAMAP2, WISDM, HHAR, HARTH, and UCI-HAR, HeteroWear improves mean macro-F1 from 0.688 to 0.724 over the identical architecture trained from scratch, with the largest gain on HHAR (0.823 versus 0.741), and achieves the highest macro-F1 on four of five benchmarks relative to single-accelerometer supervised baselines. Held-out-device evaluation yields a mean +0.066 macro-F1 improvement. Under 50% test-time channel removal, HeteroWear retains 73.8% of full-channel performance versus 64.1% for the same architecture trained from scratch without sensor dropout. Ablations show that structured sensor dropout primarily drives robustness, while cross-configuration consistency and sensor-context encoding contribute to transfer.

## Self-supervised learning for zebrafish embryo phenotype evaluation: Systematic comparison of algorithms, label-efficient classification, and transfer learning

DevFeed: [Self-supervised learning for zebrafish embryo phenotype evaluation: Systematic comparison of algorithms, label-efficient classification, and transfer learning](<https://devfeed.tech/articles/self-supervised-learning-for-zebrafish-embryo-phenotype-evaluation-systematic-comparison-of-algorithms-label-efficient-classification-and-transfer-learning-137556.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.20269>)

Author: Thomas Lautenschlager, Ga\\"elle Hayot, Quentin Brassart, Angelo Jovin Yamachui Sitcheu, Linus Schilpp, Tijana Geroski, Tetyana Goldanova, Stephan Allgeier, Thibaut Soubri\\'e, Milica Nikolic, Thomas Di

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Self-supervised learning](<https://devfeed.tech/topics/self-supervised-learning.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [generalization in machine learning](<https://devfeed.tech/topics/generalization-in-machine-learning.md>), [Dataset development](<https://devfeed.tech/topics/dataset-development.md>)

Tags: [benchmark](<https://devfeed.tech/tags/benchmark.md>), [classification](<https://devfeed.tech/tags/classification.md>), [comparison](<https://devfeed.tech/tags/comparison.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [dataset](<https://devfeed.tech/tags/dataset.md>), [imaging](<https://devfeed.tech/tags/imaging.md>), [self-supervised-learning](<https://devfeed.tech/tags/self-supervised-learning.md>), [supervised-learning](<https://devfeed.tech/tags/supervised-learning.md>), [transfer-learning](<https://devfeed.tech/tags/transfer-learning.md>), [zebrafish](<https://devfeed.tech/tags/zebrafish.md>)

### AI overview

On the public EmbryoNet dataset, a ResNet50 trained with SwAV learned representations that let linear and nonlinear classifiers reach 86.4% and 87.1% accuracy, respectively; the latter matched the fully supervised EmbryoNet result. With only 5% of labels, SwAV reached 70.2% accuracy versus 45.7% for supervised training from scratch. On a small, previously unseen toxicity dataset captured in a different setting, the SwAV model achieved 81.3% accuracy without retraining, compared with 75.9% for EmbryoNet, across normal, dead and falsely cropped classes. The abstract concludes that SSL appears most useful for label efficiency and transfer to new settings, rather than improving fully supervised results.

### Source excerpt

arXiv:2611.20269v1 Announce Type: cross Abstract: Zebrafish embryos are a prominent model organism for image-based phenotyping and especially suited for large-scale imaging studies, for which machine learning (ML) models can automate evaluation. However, training ML models usually requires large labeled datasets, and differences in imaging conditions often impede reuse across laboratories. Self-supervised learning (SSL) addresses both constraints by learning transferable representations from unlabeled images. Yet central aspects like the choice of algorithm remain underexplored for domain-specific applications. To close this gap, we benchmark popular SSL algorithms on the public EmbryoNet dataset, explore label-efficient learning, and test transfer to an unseen setting. Using representations from our strongest model, a ResNet50 network trained with SwAV, a simple linear classifier reaches 86.4% accuracy and a non-linear classifier 87.1%, rivaling the fully supervised EmbryoNet model (87.1%). The SwAV model decisively outperforms supervised training from scratch when labels are scarce (70.2% vs. 45.7% accuracy with 5% of labels). On a three-class problem (normal, dead, falsely cropped) with a small, previously unseen zebrafish toxicity dataset, recorded in a different setting, our model reaches 81.3% accuracy without retraining, while EmbryoNet achieves 75.9%. Overall, self-supervised models for zebrafish embryos seem to mostly facilitate label-efficiency and transfer to new settings rather than improving supervised results.

## Learning What the True Labels Would Teach: Posterior Analytic Class-Incremental Learning with Unlabeled Data

DevFeed: [Learning What the True Labels Would Teach: Posterior Analytic Class-Incremental Learning with Unlabeled Data](<https://devfeed.tech/articles/learning-what-the-true-labels-would-teach-posterior-analytic-class-incremental-learning-with-unlabeled-data-139110.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.20937>)

Author: Wei Tang, Tong Wei, Jiahao Zhang, Min-Ling Zhang, Lijie Hu, Fakhri Karray

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Self-supervised learning](<https://devfeed.tech/topics/self-supervised-learning.md>), [Inference](<https://devfeed.tech/topics/inference.md>), [Image](<https://devfeed.tech/topics/image.md>)

Tags: [best](<https://devfeed.tech/tags/best.md>), [classifier](<https://devfeed.tech/tags/classifier.md>), [clip](<https://devfeed.tech/tags/clip.md>), [cross-validation](<https://devfeed.tech/tags/cross-validation.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [data](<https://devfeed.tech/tags/data.md>), [distribution](<https://devfeed.tech/tags/distribution.md>), [expectation](<https://devfeed.tech/tags/expectation.md>), [features](<https://devfeed.tech/tags/features.md>), [fusion](<https://devfeed.tech/tags/fusion.md>), [gap](<https://devfeed.tech/tags/gap.md>), [incremental](<https://devfeed.tech/tags/incremental.md>), [learning](<https://devfeed.tech/tags/learning.md>), [name](<https://devfeed.tech/tags/name.md>), [prediction](<https://devfeed.tech/tags/prediction.md>), [train](<https://devfeed.tech/tags/train.md>), [training](<https://devfeed.tech/tags/training.md>), [weight](<https://devfeed.tech/tags/weight.md>), [zero-shot](<https://devfeed.tech/tags/zero-shot.md>)

### AI overview

PACIL addresses unlabeled class-incremental learning by using soft label posteriors rather than hard pseudo-labels, which can assign zero probability to the true class after an error. It corrects class priors and refines image distributions with a fixed-prior von Mises-Fisher mixture on whitened features; these soft targets train an analytic classifier and support closed-form leave-one-out selection of its fusion weight with zero-shot CLIP. The authors report theoretical results on posterior targets and fusion-weight selection, and the best average incremental accuracy among methods trained without labels across all 18 evaluated settings. The supplied text is an abstract, so it does not provide implementation details or per-setting results.

### Source excerpt

arXiv:2611.20937v1 Announce Type: new Abstract: In class-incremental learning with unlabeled data, each task brings only class names and unlabeled images, so CLIP's zero-shot predictions replace labels. Existing methods train on hard pseudo-labels, but a wrong one gives the true class zero probability, which self-training often confirms. We argue that each image should carry its label posterior, as an analytic classifier trained on posteriors equals the true-label one in expectation. The zero-shot prediction is not this posterior, as its class proportions confound name bias with true imbalance and it ignores other images. We propose Posterior Analytic Class-Incremental Learning (PACIL), which corrects the class prior with a prior selection rule and refines each distribution by a fixed-prior von Mises-Fisher mixture on whitened features. These soft labels train the classifier and replace labels to select its fusion weight with zero-shot CLIP by closed-form leave-one-out cross-validation. We prove that posterior targets minimize the mean-square gap to the true-label prediction and that their argmax widens this gap only through ambiguous images. The fusion weight selected with soft labels loses at most their distance to the posteriors in expected correct decisions. PACIL has the best average incremental accuracy among methods trained without labels on all 18 settings.

## PINR-ESI: Physics-Informed Self-Supervised Implicit Neural Representation Framework for EEG Extended-Source Imaging

DevFeed: [PINR-ESI: Physics-Informed Self-Supervised Implicit Neural Representation Framework for EEG Extended-Source Imaging](<https://devfeed.tech/articles/pinr-esi-physics-informed-self-supervised-implicit-neural-representation-framework-for-eeg-extended-source-imaging-135741.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.06700>)

Author: Zhiyong Zhou, Sheng Liao, Xiaolin Li, Youshen Xiao, Bin Dai, Surui Liu, Yakang Dai

Published: 2026-11-16T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Self-supervised learning](<https://devfeed.tech/topics/self-supervised-learning.md>)

Tags: [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [eeg](<https://devfeed.tech/tags/eeg.md>), [imaging](<https://devfeed.tech/tags/imaging.md>), [neural](<https://devfeed.tech/tags/neural.md>), [physics](<https://devfeed.tech/tags/physics.md>), [physics-informed](<https://devfeed.tech/tags/physics-informed.md>), [q-bio-nc](<https://devfeed.tech/tags/q-bio-nc.md>), [source](<https://devfeed.tech/tags/source.md>)

### AI overview

PINR-ESI reconstructs extended cortical EEG sources without paired source labels or simulation-based supervised pretraining. It represents cortical current density as a continuous coordinate-based neural field, samples it at cortical vertices, applies local smoothing, and maps the result to predicted EEG with the lead-field matrix. Agreement between predicted and measured EEG constrains the reconstruction; a classical inverse estimate provides warm initialization and observation-specific spatial guidance through prior-guided support regularization. Simulations across different SNRs, source extents, and source counts reported low spatial dispersion and good localization. Clinical epilepsy EEG evaluation and qualitative analysis during intracranial stimulation supported feasibility on real-world data. The supplied abstract does not give detailed metrics or comparisons.

### Source excerpt

arXiv:2611.06700v3 Announce Type: replace Abstract: EEG extended-source imaging aims to localize active cortical regions and characterize their extent, yet supervised inverse networks typically depend on synthetic EEG-source pairs and may be sensitive to synthetic-to-real mismatch. We propose a physics-informed self-supervised implicit neural representation (INR) framework for spatial source reconstruction without paired source labels or simulation-based supervised pretraining. The lead-field matrix encodes the source-to-sensor volume-conduction model and constrains the INR through agreement between predicted and measured EEG. Cortical current density is represented as a continuous coordinate-based neural field. Forward EEG prediction samples the field at cortical vertices and applies local smoothing before lead-field mapping. A classical inverse estimate provides warm initialization and observation-specific spatial guidance through prior-guided support regularization. Simulations across varying SNRs, source extents, and source counts showed low spatial dispersion and good localization performance. Quantitative evaluation on clinical EEG recordings from epilepsy patients and qualitative analysis of scalp EEG during intracranial stimulation supported feasibility for real-world source imaging. These findings suggest the potential of physics-informed self-supervised INR for extended-source imaging.

## Contrastive latent representation learning with denoising score matching

DevFeed: [Contrastive latent representation learning with denoising score matching](<https://devfeed.tech/articles/contrastive-latent-representation-learning-with-denoising-score-matching-135907.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.18469>)

Author: Christopher Marouani, Grigory Bartosh, Christian A. Naesseth

Published: 2026-11-16T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Representation learning](<https://devfeed.tech/topics/representation-learning.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Self-supervised learning](<https://devfeed.tech/topics/self-supervised-learning.md>)

Tags: [batch](<https://devfeed.tech/tags/batch.md>), [challenge](<https://devfeed.tech/tags/challenge.md>), [cifar-10](<https://devfeed.tech/tags/cifar-10.md>), [cost](<https://devfeed.tech/tags/cost.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [curse-of-dimensionality](<https://devfeed.tech/tags/curse-of-dimensionality.md>), [denoising](<https://devfeed.tech/tags/denoising.md>), [entropy](<https://devfeed.tech/tags/entropy.md>), [gaussian](<https://devfeed.tech/tags/gaussian.md>), [gradient](<https://devfeed.tech/tags/gradient.md>), [information](<https://devfeed.tech/tags/information.md>), [learning](<https://devfeed.tech/tags/learning.md>), [lightweight](<https://devfeed.tech/tags/lightweight.md>), [linear](<https://devfeed.tech/tags/linear.md>), [matching](<https://devfeed.tech/tags/matching.md>), [network](<https://devfeed.tech/tags/network.md>), [noise](<https://devfeed.tech/tags/noise.md>), [novel](<https://devfeed.tech/tags/novel.md>), [rank](<https://devfeed.tech/tags/rank.md>), [representation-learning](<https://devfeed.tech/tags/representation-learning.md>), [score](<https://devfeed.tech/tags/score.md>), [stat-ml](<https://devfeed.tech/tags/stat-ml.md>)

### AI overview

ScoreCLR estimates contrastive repulsion by learning the score of latent representations. It derives an objective from the gradient of Shannon entropy with respect to encoder parameters, then estimates the score using a lightweight denoising score matching network trained jointly with the encoder. Regularization uses multiple noise levels and a Gaussian prior. This replaces explicit negative samples: the reported repulsion cost scales linearly rather than quadratically with batch size. On CIFAR-10 and ImageNet-100, the abstract reports minimal linear-probe accuracy loss versus SimCLR under a shared recipe, with projector outputs of higher effective rank and closer to the Gaussian prior.

### Source excerpt

arXiv:2611.18469v1 Announce Type: cross Abstract: Contrastive representation learning methods operate by pulling different views of the same data sample together while pushing them away from views of different samples. A fundamental challenge is the estimation of the repulsive force, which typically requires large batch sizes, can exhibit high variance, and is susceptible to the curse of dimensionality. We introduce ScoreCLR, a contrastive representation learning method that offers a novel way to estimate this repulsion term. We consider the framework of representation learning by maximizing the mutual information between two latent views of a sample. By deriving the gradient of the Shannon entropy with respect to the encoder parameters, we obtain an equivalent objective that involves the score of the latent representation. We then use denoising score matching techniques to estimate this score with a lightweight network that is trained jointly with the encoder. We further propose regularization techniques using different noise levels and a Gaussian prior on the latent representation. ScoreCLR replaces the batch of explicit negative samples in contrastive methods with a learned repulsion whose cost is linear in the batch size rather than quadratic. On CIFAR-10 and ImageNet-100 it does so with minimal loss of linear-probe accuracy compared to SimCLR under a shared recipe, while producing projector outputs of higher effective rank, closer to the Gaussian prior.

## BAL: Bidirectional Autoregression in Latent Space for Learning Human Movement Representations

DevFeed: [BAL: Bidirectional Autoregression in Latent Space for Learning Human Movement Representations](<https://devfeed.tech/articles/bal-bidirectional-autoregression-in-latent-space-for-learning-human-movement-representations-135919.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.18559>)

Author: Genki Kinoshita, Ko Nishino

Published: 2026-11-16T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Representation learning](<https://devfeed.tech/topics/representation-learning.md>), [Self-supervised learning](<https://devfeed.tech/topics/self-supervised-learning.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>)

Tags: [3d](<https://devfeed.tech/tags/3d.md>), [autoregressive](<https://devfeed.tech/tags/autoregressive.md>), [behavior](<https://devfeed.tech/tags/behavior.md>), [context](<https://devfeed.tech/tags/context.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [diversity](<https://devfeed.tech/tags/diversity.md>), [forecasting](<https://devfeed.tech/tags/forecasting.md>), [framework](<https://devfeed.tech/tags/framework.md>), [future](<https://devfeed.tech/tags/future.md>), [global](<https://devfeed.tech/tags/global.md>), [learning](<https://devfeed.tech/tags/learning.md>), [local](<https://devfeed.tech/tags/local.md>), [ordering](<https://devfeed.tech/tags/ordering.md>), [prediction](<https://devfeed.tech/tags/prediction.md>), [sequence](<https://devfeed.tech/tags/sequence.md>), [sequences](<https://devfeed.tech/tags/sequences.md>), [signal](<https://devfeed.tech/tags/signal.md>), [supervision](<https://devfeed.tech/tags/supervision.md>), [target](<https://devfeed.tech/tags/target.md>), [temporal](<https://devfeed.tech/tags/temporal.md>)

### AI overview

BAL learns human movement representations from 3D pose sequences by predicting future and past states in an abstract latent space, using natural temporal ordering as self-supervision instead of heuristic augmentation or masking assumptions. At inference, it can use both past and future context. To reduce representation collapse, its prediction targets combine a sequence-level global component with a context-independent local component; the abstract says the local targets retain angular diversity unless representations fully collapse. On large-scale datasets, BAL reportedly outperforms baselines on most metrics for motion captioning, forecasting, and interpolation.

### Source excerpt

arXiv:2611.18559v1 Announce Type: cross Abstract: Learning semantically meaningful human movement representations from 3D pose sequences is essential for human behavior modeling. Existing self-supervised methods derive supervision from artificial assumptions, e.g., invariance to augmentation within a certain range or recoverability of masked regions at a certain temporal scale, introducing heuristic hyperparameters that make the learned semantics sensitive to their settings. We propose BAL, a self-supervised framework that instead exploits supervision intrinsic to human movement: the natural ordering structure of everyday behavior. BAL autoregressively predicts in a semantically abstract latent space, analogous to how humans anticipate others' actions rather than their precise joint configurations. Bidirectional autoregressive prediction, both forward and backward in time, fully leverages these ordering constraints while enabling representation extraction using both past and future context at inference. To suppress representation collapse, BAL decomposes each prediction target into a global component capturing sequence-level semantics and a local component capturing context-independent movement. We prove that the local targets retain nontrivial angular diversity unless the representations are completely collapsed, preserving a meaningful training signal. Experiments on large-scale datasets demonstrate that BAL's representations are highly effective for motion captioning, forecasting, and interpolation, outperforming baselines on most evaluation metrics.

## Contrastive Bootstrap Representation Learning for Label-Efficient Agricultural Image Classification

DevFeed: [Contrastive Bootstrap Representation Learning for Label-Efficient Agricultural Image Classification](<https://devfeed.tech/articles/contrastive-bootstrap-representation-learning-for-label-efficient-agricultural-image-classification-132153.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.15610>)

Author: Hazem Ahmed, Ali Hamdi, Ammar Mohammed

Published: 2026-11-13T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Representation learning](<https://devfeed.tech/topics/representation-learning.md>), [Self-supervised learning](<https://devfeed.tech/topics/self-supervised-learning.md>), [image recognition](<https://devfeed.tech/topics/image-recognition.md>), [Image](<https://devfeed.tech/topics/image.md>)

Tags: [annotation](<https://devfeed.tech/tags/annotation.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [bootstrap](<https://devfeed.tech/tags/bootstrap.md>), [classification](<https://devfeed.tech/tags/classification.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [image-classification](<https://devfeed.tech/tags/image-classification.md>), [learning](<https://devfeed.tech/tags/learning.md>), [pretraining](<https://devfeed.tech/tags/pretraining.md>), [self-supervised-learning](<https://devfeed.tech/tags/self-supervised-learning.md>), [supervised-learning](<https://devfeed.tech/tags/supervised-learning.md>)

### AI overview

CBRL combines SimCLR-style NT-Xent contrastive learning with bidirectional BYOL-style exponential-moving-average target prediction in one online projection space. With a ViT-L/16 backbone and 10% of downstream labels, it achieved the highest reported macro-F1 point estimate on four primary agricultural benchmarks in single-seed evaluations. DeepWeeds reached 79.77%, 15.78 points above the next-highest comparison; PlantVillage reached 98.14%, 0.43 points above it. On PlantDoc, no method reached practically useful performance. The results suggest promise for label-limited transfer, but do not establish objective synergy, statistical superiority, or robustness across runs.

### Source excerpt

arXiv:2611.15610v1 Announce Type: cross Abstract: Expert annotation remains a major bottleneck in agricultural image classification, where collecting images is often easier than obtaining reliable labels from domain specialists. Self-supervised learning can reduce this dependence by pretraining transferable representations on unlabelled imagery. Contrastive and bootstrap methods provide two established but distinct training signals, motivating an empirical study of a formulation that applies both signals within one representation path under severe downstream label scarcity. We present Contrastive Bootstrap Representation Learning (CBRL), which jointly optimises SimCLR-style NT-Xent contrastive learning and bidirectional BYOL-style exponential-moving-average target prediction within a shared online projection space. Using a ViT-L/16 backbone and 10% of the downstream labels, we report single-seed point estimates on four primary agricultural test benchmarks plus an exploratory 33-image New Plant Diseases evaluation. CBRL records the highest macro-F1 point estimate on the four primary benchmarks. On DeepWeeds, its 79.77% test macro-F1 is 32.29 points above the random-initialisation, label-only baseline and 15.78 points above the next-highest comparison point estimate; on PlantVillage, its 98.14% macro-F1 is 10.43 and 0.43 points higher, respectively. On PlantDoc, however, all methods fail to reach practically useful performance. These results identify the evaluated joint formulation as a promising candidate for label-limited transfer, but they do not establish objective synergy, statistical superiority, or robustness across stochastic runs.

## Spectral Ghost in Representation Learning: A Unified View from Component Analysis to Self-Supervised Learning

DevFeed: [Spectral Ghost in Representation Learning: A Unified View from Component Analysis to Self-Supervised Learning](<https://devfeed.tech/articles/spectral-ghost-in-representation-learning-a-unified-view-from-component-analysis-to-self-supervised-learning-132742.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2601.20154>)

Author: Bo Dai, Na Li, Dale Schuurmans

Published: 2026-11-13T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Representation learning](<https://devfeed.tech/topics/representation-learning.md>), [Self-supervised learning](<https://devfeed.tech/topics/self-supervised-learning.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>)

Tags: [algorithms](<https://devfeed.tech/tags/algorithms.md>), [causal-inference](<https://devfeed.tech/tags/causal-inference.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [range](<https://devfeed.tech/tags/range.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [self-supervised-learning](<https://devfeed.tech/tags/self-supervised-learning.md>), [ssl](<https://devfeed.tech/tags/ssl.md>), [sufficient](<https://devfeed.tech/tags/sufficient.md>)

### AI overview

The paper proposes a unified spectral framework in which a representation is sufficient for arbitrary downstream prediction with a linear head when it spans the spectral decomposition of pointwise mutual information, which the authors say can be extracted from unlabeled data. It organizes representation methods by the form of the spectral representation and the objective used to fit it, relating a range of self-supervised and classical methods to combinations of those choices. It also argues that batch-size requirements depend on whether an objective is unbiased, a consistent surrogate, or biased, rather than on whether its loss is contrastive. The abstract says the framework motivates scalable algorithms and extensions to causal inference and reinforcement learning; it provides no details about those algorithms or extensions.

### Source excerpt

arXiv:2601.20154v4 Announce Type: replace Abstract: Self-supervised learning (SSL) extracts representations from massive unlabeled data that transfer to a variety of downstream tasks. Its empirical success has produced a dizzying array of objectives and procedures, yet a clear understanding of what representation is desired, and why these seemingly different methods work, is still missing. In this paper, we develop a unified spectral framework for representation learning. We first show that a representation is sufficient for arbitrary downstream prediction with a linear head if it spans the spectral decomposition of the pointwise mutual information, which can be extracted purely from unlabeled data. This view organizes representation learning along two axes, the parametrization of the spectral representation (linear, energy-based, denoising, nonlinear, latent-variable, and multi-modal) and the learning objective for fitting it, under which a broad range of existing methods, including SimCLR, MoCo, BYOL, Barlow Twins, VICReg, DINO, MAE, JEPA, BERT, GPT, and CLIP, as well as classic PCA, CCA, Laplacian eigenmaps, LLE, and SNE, are recovered as particular combinations. The framework also clarifies optimization behaviors: the batch-size requirement is determined by whether the objective is unbiased, a consistent surrogate, or biased, rather than by contrastive vs. non-contrastive losses. Finally, the framework inspires new scalable algorithms and extends spectral representations to causal inference and reinforcement learning.

## Self-Modeling Interventions can Modulate Emergent Misalignment

DevFeed: [Self-Modeling Interventions can Modulate Emergent Misalignment](<https://devfeed.tech/articles/self-modeling-interventions-can-modulate-emergent-misalignment-128124.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2606.23700>)

Author: Arush Tagade, Shaoheng Zhou, Jiaxin Wen, Shi Feng

Published: 2026-11-11T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [AI Chat](<https://devfeed.tech/topics/ai-chat.md>), [Self-supervised learning](<https://devfeed.tech/topics/self-supervised-learning.md>)

Tags: [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [defense](<https://devfeed.tech/tags/defense.md>), [em](<https://devfeed.tech/tags/em.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [fragmentation](<https://devfeed.tech/tags/fragmentation.md>), [identity](<https://devfeed.tech/tags/identity.md>), [llm](<https://devfeed.tech/tags/llm.md>), [misalignment](<https://devfeed.tech/tags/misalignment.md>), [modeling](<https://devfeed.tech/tags/modeling.md>), [persona](<https://devfeed.tech/tags/persona.md>)

### AI overview

Across GPT-4.1, Qwen2.5-32B-Instruct, and Seed-OSS-36B-Instruct, self-recognition fine-tuning reduced emergent misalignment in reversal and prevention experiments. In prevention, it was the only tested fine-tuning approach that consistently reduced misalignment without worsening an individual metric; comparisons included domain-specific data, general knowledge, and word-counting baselines. Separate in-training experiments compared self-reports with benign interleaving and inoculation prompting. The abstract also reports that EM fine-tuning fragmented identity self-reports and reduced self-recognition to about chance, while corrupting self-recognition worsened misalignment and training on self-reports from EM-tuned models transferred it. These results support the authors' interpretation that EM destabilizes aligned character; the supplied abstract does not provide effect sizes or detailed experimental procedures.

### Source excerpt

arXiv:2606.23700v2 Announce Type: replace-cross Abstract: Emergent misalignment (EM) has been linked to the activation of misaligned persona vectors and evil character traits, suggesting that EM operates through disruption of the model's aligned character rather than direct learning of harmful content. Motivated by this connection, we study self-modeling interventions: specifically self-recognition and self-reports as character-targeted interventions that are distinct from existing in-training defenses. We conduct two-stage finetuning experiments across three models (GPT-4.1, Qwen2.5-32B-Instruct, Seed-OSS-36B-Instruct) and multiple EM datasets to compare self-recognition finetuning against benign finetuning baselines (correct domain-specific data, general knowledge, and word counting), finding it an effective defense in both reversal and prevention settings. For prevention, only self-recognition finetuning consistently reduces misalignment without exacerbating any individual metric, suggesting that character fortification specifically drives prevention. We also conduct in-training experiments across these models to compare the effectiveness of self-reports with benign interleaving and inoculation prompting. We provide further evidence for EM's relation to the LLM's default character by showing that EM finetuning leads to fragmentation of LLM's identity self-reports, reduces self-recognition performance to about chance, artificially corrupting self-recognition exacerbates misalignment caused by EM finetuning, and finetuning on self-reports from EM finetuned models can transfer misalignment. Together, these findings reframe EM not as the adoption of a coherent misaligned persona but as the destabilization of aligned character.

## Reasoning-Aware Relational Representation Learning for Open-Vocabulary Scene Graph Generation

DevFeed: [Reasoning-Aware Relational Representation Learning for Open-Vocabulary Scene Graph Generation](<https://devfeed.tech/articles/reasoning-aware-relational-representation-learning-for-open-vocabulary-scene-graph-generation-128319.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.13496>)

Author: Jiawei Xiao, Boya Wang, Longtian Qiu, Shan Ning, Xuming He

Published: 2026-11-11T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Representation learning](<https://devfeed.tech/topics/representation-learning.md>), [Embeddings](<https://devfeed.tech/topics/embeddings.md>), [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [image recognition](<https://devfeed.tech/topics/image-recognition.md>), [Self-supervised learning](<https://devfeed.tech/topics/self-supervised-learning.md>)

Tags: [chain-of-thought](<https://devfeed.tech/tags/chain-of-thought.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [dataset](<https://devfeed.tech/tags/dataset.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [embeddings](<https://devfeed.tech/tags/embeddings.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [framework](<https://devfeed.tech/tags/framework.md>), [generation](<https://devfeed.tech/tags/generation.md>), [graph](<https://devfeed.tech/tags/graph.md>), [learning](<https://devfeed.tech/tags/learning.md>), [localization](<https://devfeed.tech/tags/localization.md>), [models](<https://devfeed.tech/tags/models.md>), [object](<https://devfeed.tech/tags/object.md>), [open](<https://devfeed.tech/tags/open.md>), [patterns](<https://devfeed.tech/tags/patterns.md>), [predicate](<https://devfeed.tech/tags/predicate.md>), [prediction](<https://devfeed.tech/tags/prediction.md>), [reasoning](<https://devfeed.tech/tags/reasoning.md>), [recognition](<https://devfeed.tech/tags/recognition.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [relationships](<https://devfeed.tech/tags/relationships.md>)

### AI overview

RaRe targets biased relation representations in open-vocabulary scene graph generation, which can hinder rare-relation recognition and generalization to unseen predicates. It turns MLLM-generated chain-of-thought rationales into structured relation embeddings, aligns them with textual relation descriptions using self-supervised contrastive learning, and refines rationale generation with a reinforcement-learning objective that favors evidence specific to each object pair. Experiments on VG150 and PSG report improved category-balanced recognition over existing baselines; additional gains vary across datasets and proposal settings.

### Source excerpt

arXiv:2611.13496v1 Announce Type: cross Abstract: Open-Vocabulary Scene Graph Generation (OV-SGG) requires models to recognize visual relationships beyond the training vocabulary, yet existing methods often rely on dataset-specific object--predicate co-occurrence patterns. Under long-tailed distributions and noisy object localization, such reliance leads to biased, entangled relation representations, limiting both rare-relation recognition and generalization to unseen predicates. To address these challenges, we propose RaRe, a reasoning-aware relational representation learning framework that transforms MLLM-generated chain-of-thought (CoT) rationales into structured relation embeddings. Rather than using CoT only as an intermediate reasoning trace for final prediction, RaRe directly optimizes the rationale-derived representation space to improve inter-relation discriminability. Specifically, RaRe first aligns rationale embeddings with textualized relation descriptions through self-supervised contrastive learning, and then refines rationale generation with a specificity-oriented reinforcement learning objective that encourages semantically object-pair-specific relational evidence. Experiments on VG150 and PSG demonstrate improved category-balanced recognition over existing baselines, while the additional gains from learning vary across datasets and proposal settings. Code available at: https://artanic30.github.io/project_pages/RaRe.

## Deviation-Aware Pretraining for Cardiovascular Disease Detection from Electrocardiograms

DevFeed: [Deviation-Aware Pretraining for Cardiovascular Disease Detection from Electrocardiograms](<https://devfeed.tech/articles/deviation-aware-pretraining-for-cardiovascular-disease-detection-from-electrocardiograms-128807.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.14141>)

Author: Siyuan Yan, Haibo Zheng, Feng Lu, Wei Li

Published: 2026-11-11T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Self-supervised learning](<https://devfeed.tech/topics/self-supervised-learning.md>), [Embeddings](<https://devfeed.tech/topics/embeddings.md>)

Tags: [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [detection](<https://devfeed.tech/tags/detection.md>), [disease](<https://devfeed.tech/tags/disease.md>), [ecg](<https://devfeed.tech/tags/ecg.md>), [embedding](<https://devfeed.tech/tags/embedding.md>), [feature](<https://devfeed.tech/tags/feature.md>), [healthcare](<https://devfeed.tech/tags/healthcare.md>), [learning](<https://devfeed.tech/tags/learning.md>), [monitoring](<https://devfeed.tech/tags/monitoring.md>), [pretraining](<https://devfeed.tech/tags/pretraining.md>)

### AI overview

The framework pretrains ECG features without labels on nearly one million 12-lead recordings. Deviation Localization Embedding (DLE) detects waveform deviations from normal patterns, while Association Reconstruction Embedding (ARE) uses deviation-guided reconstruction to model cross-lead associations. On multiple public datasets, the approach reportedly improves AUROC by up to 6.5% and F1 by up to 12.8% over prior methods, even without further fine-tuning the pretrained feature extractor. The abstract also reports disease grouping and anomaly identification in embedding visualizations; it does not provide dataset names or evaluation details.

### Source excerpt

arXiv:2611.14141v1 Announce Type: new Abstract: Electrocardiogram (ECG) signals are critical for cardiovascular disease (CVD) detection. Most current deep learning models rely on supervised, task-specific training, which limits their ability to generalize across diseases, patient populations, and acquisition settings. This paper presents a self-supervised ECG feature extraction framework that learns from nearly one million unlabeled 12-lead ECG recordings. The framework includes two modules: Deviation Localization Embedding (DLE), which detects waveform-level deviations from normal patterns, and Association Reconstruction Embedding (ARE), which models cross-lead associations through deviation-guided reconstruction to capture abnormal patterns expressed across multiple leads. These modules enable the framework to capture subtle deviation-related patterns and their cross-lead context. The framework achieves up to 6.5% AUROC and 12.8% F1 improvements over prior methods on multiple public datasets, even without additional fine-tuning the pretrained feature extractor. Embedding visualizations show that the learned features support meaningful disease grouping and anomaly identification. The approach reduces the need for labeled data and can support clinicians in diverse healthcare settings. The framework has potential to improve early detection and monitoring of cardiovascular diseases, contributing to more accessible and equitable heart care.

[Next page](<https://devfeed.tech/topics/self-supervised-learning.md?cursor=WyIyMDI2LTExLTExVDA1OjAwOjAwKzAwOjAwIiwgIjUwMzJmZWYyLWI1NGYtNDAyYy1hYTQxLTBlODAwZmJkMGVjMCJd>)