# DenseTwins: Dense Modality Decoupling for Multimodal Self-Supervision

DevFeed: [DenseTwins: Dense Modality Decoupling for Multimodal Self-Supervision](<https://devfeed.tech/articles/densetwins-dense-modality-decoupling-for-multimodal-self-supervision-116975.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.04710>)

Author: Carlo Sgaravatti, Daniele Kopyshevskiy, Davide Secco, Giacomo Boracchi

Published: 2026-11-04T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Representation learning](<https://devfeed.tech/topics/representation-learning.md>), [multimodal-ai](<https://devfeed.tech/topics/multimodal-ai.md>)

Tags: [alignment](<https://devfeed.tech/tags/alignment.md>), [applications](<https://devfeed.tech/tags/applications.md>), [autonomous-driving](<https://devfeed.tech/tags/autonomous-driving.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [decoupling](<https://devfeed.tech/tags/decoupling.md>), [embedding](<https://devfeed.tech/tags/embedding.md>), [features](<https://devfeed.tech/tags/features.md>), [framework](<https://devfeed.tech/tags/framework.md>), [multimodal](<https://devfeed.tech/tags/multimodal.md>), [objectives](<https://devfeed.tech/tags/objectives.md>), [performance](<https://devfeed.tech/tags/performance.md>), [reduction](<https://devfeed.tech/tags/reduction.md>), [redundancy](<https://devfeed.tech/tags/redundancy.md>), [robotics](<https://devfeed.tech/tags/robotics.md>), [self-supervised-learning](<https://devfeed.tech/tags/self-supervised-learning.md>), [semantics](<https://devfeed.tech/tags/semantics.md>), [sensing](<https://devfeed.tech/tags/sensing.md>), [space](<https://devfeed.tech/tags/space.md>), [spatial](<https://devfeed.tech/tags/spatial.md>), [supervision](<https://devfeed.tech/tags/supervision.md>)

## AI overview

DenseTwins learns dense multimodal representations by separating shared features from modality-specific ones, aiming to preserve both cross-modal semantics and spatial structure. It combines dense cross-modal correspondences with redundancy-reduction objectives: an intra-modal objective connects augmented views, while a cross-modal objective aligns shared features and decorrelates modality-specific information. The abstract reports state-of-the-art or competitive performance against multimodal self-supervised baselines across multiple benchmarks and sensing setups; it provides no benchmark-specific results or further implementation details.

## Source excerpt

arXiv:2611.04710v1 Announce Type: cross Abstract: Multimodal perception is central to many computer vision applications, including robotics and autonomous driving, where different sensing modalities provide complementary cues. Self-supervised learning offers a scalable solution for exploiting large amounts of unlabeled multimodal data. However, existing methods either neglect modality-specific information by enforcing feature-space alignment between modalities or learn only global representations without preserving spatial structure. We propose DenseTwins, a multimodal self-supervised framework that learns dense representations by explicitly decoupling the dense feature space into shared and modality-specific components. The proposed approach combines dense cross-modal correspondences with redundancy-reduction objectives, enabling the model to capture both shared semantics and modality-specific cues within a structured dense embedding space. DenseTwins jointly optimizes an intra-modal objective across augmented views and a cross-modal objective that aligns shared features while decorrelating modality-specific information. We evaluate DenseTwins across multiple multimodal benchmarks and sensing setups, achieving state-of-the-art or competitive performance across multimodal self-supervised baselines. Code available at: https://github.com/CarloSgaravatti/DenseTwins.