# Adaptive Dual-Teacher Distillation with Subnetwork Rectification for Bridging Semantic Gaps in Black-Box Domain Adaptation

DevFeed: [Adaptive Dual-Teacher Distillation with Subnetwork Rectification for Bridging Semantic Gaps in Black-Box Domain Adaptation](<https://devfeed.tech/articles/adaptive-dual-teacher-distillation-with-subnetwork-rectification-for-bridging-semantic-gaps-in-black-box-domain-adaptation-94115.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2603.22908>)

Author: Zhe Zhang, Jing Li, Wanli Xue, Xu Cheng, Jianhua Zhang, Qinghua Hu, Shengyong Chen

Published: 2026-10-21T04:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Post-training optimization](<https://devfeed.tech/topics/post-training-optimization.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [accessible](<https://devfeed.tech/tags/accessible.md>), [adaptive](<https://devfeed.tech/tags/adaptive.md>), [alignment](<https://devfeed.tech/tags/alignment.md>), [art](<https://devfeed.tech/tags/art.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [based](<https://devfeed.tech/tags/based.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [black](<https://devfeed.tech/tags/black.md>), [box](<https://devfeed.tech/tags/box.md>), [bridging](<https://devfeed.tech/tags/bridging.md>), [but](<https://devfeed.tech/tags/but.md>), [challenge](<https://devfeed.tech/tags/challenge.md>), [class](<https://devfeed.tech/tags/class.md>), [consistency](<https://devfeed.tech/tags/consistency.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [data](<https://devfeed.tech/tags/data.md>), [distillation](<https://devfeed.tech/tags/distillation.md>), [domain](<https://devfeed.tech/tags/domain.md>), [semantic](<https://devfeed.tech/tags/semantic.md>), [with](<https://devfeed.tech/tags/with.md>)

## AI overview

DDSR targets black-box domain adaptation when source data and model parameters are unavailable, using only source-model predictions. It combines those predictions with a vision-language model (ViL) through adaptive fusion to produce target-domain pseudo-labels. Subnetwork regularization is intended to limit overfitting to noisy supervision by enforcing output consistency and gradient divergency. As target predictions improve, they iteratively refine both pseudo-labels and ViL prompts; class-wise prototypes then support self-training. The abstract reports that DDSR outperforms state-of-the-art methods across multiple benchmarks, including methods with access to source data or parameters; it provides no numerical results or benchmark details.

## Source excerpt

arXiv:2603.22908v5 Announce Type: replace-cross Abstract: Assuming that neither source data nor source model parameters are accessible, black-box domain adaptation (BBDA) represents a highly practical yet challenging setting, where transferable knowledge is limited to the predictions of a black-box source model. Existing approaches exploit such knowledge via pseudo-label refinement or by leveraging vision-language models (ViLs), but they often fail to reconcile the inherent discrepancy between task-specific knowledge from black-box models and language-aligned semantic priors of ViLs, resulting in suboptimal integration and degraded adaptation performance. To address this challenge, we propose adaptive Dual-Teacher Distillation with Subnetwork Rectification (DDSR), a framework that explicitly reconciles these complementary yet inconsistent knowledge sources. DDSR employs an adaptive prediction fusion strategy to integrate predictions from the black-box source model and a ViL, generating reliable pseudo-labels for the target domain. A subnetwork-based regularization mechanism mitigates overfitting to noisy supervision by enforcing output consistency and gradient divergency. Furthermore, progressively improved target predictions iteratively refine both pseudo-labels and ViL prompts, enhancing semantic alignment. Finally, class-wise prototypes are used to further optimize target predictions via self-training. Extensive experiments on multiple benchmark datasets demonstrate that DDSR consistently outperforms state-of-the-art methods, including those with access to source data or source model parameters.