# Hidden not Deleted: How Networks Suppress Entangled Features

DevFeed: [Hidden not Deleted: How Networks Suppress Entangled Features](<https://devfeed.tech/articles/hidden-not-deleted-how-networks-suppress-entangled-features-135006.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2609.27593>)

Author: Akash Samanta, Manish Pratap Singh, Debasis Chaudhuri

Published: 2026-11-16T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Neural Network](<https://devfeed.tech/topics/neural-network.md>), [Representation learning](<https://devfeed.tech/topics/representation-learning.md>)

Tags: [activation](<https://devfeed.tech/tags/activation.md>), [analysis](<https://devfeed.tech/tags/analysis.md>), [autoencoder](<https://devfeed.tech/tags/autoencoder.md>), [bias](<https://devfeed.tech/tags/bias.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [detection](<https://devfeed.tech/tags/detection.md>), [features](<https://devfeed.tech/tags/features.md>), [flexible](<https://devfeed.tech/tags/flexible.md>), [information](<https://devfeed.tech/tags/information.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [networks](<https://devfeed.tech/tags/networks.md>), [predictions](<https://devfeed.tech/tags/predictions.md>), [recovery](<https://devfeed.tech/tags/recovery.md>), [training](<https://devfeed.tech/tags/training.md>)

## AI overview

In a small autoencoder, a silent output does not by itself show that a feature has been forgotten. With the representation fixed, restoring one bias partially recovers target detection by bringing a signal hidden below the output activation threshold into view; target-absent tests help distinguish this from simply raising output. The intervention is less effective when the representation can change. Training other outputs alone can also weaken the target contribution, which a conditional theoretical analysis links to interference with retained predictions. More flexible predictors still detect residual target information. Recovery therefore depends on feature geometry and which parameters are trained.

## Source excerpt

arXiv:2609.27593v2 Announce Type: replace-cross Abstract: When a neural network stops reporting a feature, has it forgotten the feature or merely hidden it? Shared representations complicate the answer: removing one feature's direction can also damage another feature. We study this problem in a small autoencoder that compresses and reconstructs inputs, allowing individual feature contributions to be traced directly. With the representation fixed, restoring a single bias partially recovers target detection by exposing a signal hidden behind the output's activation threshold. Tests on target-absent inputs distinguish this recovery from interventions that simply increase output. The same bias intervention becomes less effective when the representation is allowed to change. We find that training only the other outputs can weaken the target's contribution, even without a loss on its own output. A conditional theoretical analysis explains how interference with retained predictions creates this pressure. Yet more flexible predictors still detect residual target information. These findings show how recovery depends on feature geometry and on which parameters are trained, and why a silent output is insufficient evidence of forgetting.