# Neural Collapse on Imbalanced Datasets and Open-Set Scenario

DevFeed: [Neural Collapse on Imbalanced Datasets and Open-Set Scenario](<https://devfeed.tech/articles/neural-collapse-on-imbalanced-datasets-and-open-set-scenario-120662.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.07753>)

Author: Qiufu Li, Huibing Xiao, Linlin Shen, Yongdong Huang

Published: 2026-11-06T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Neural Network](<https://devfeed.tech/topics/neural-network.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [analysis](<https://devfeed.tech/tags/analysis.md>), [analyze](<https://devfeed.tech/tags/analyze.md>), [class](<https://devfeed.tech/tags/class.md>), [classes](<https://devfeed.tech/tags/classes.md>), [classification](<https://devfeed.tech/tags/classification.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [entropy](<https://devfeed.tech/tags/entropy.md>), [features](<https://devfeed.tech/tags/features.md>), [global](<https://devfeed.tech/tags/global.md>), [model](<https://devfeed.tech/tags/model.md>), [neural](<https://devfeed.tech/tags/neural.md>), [open](<https://devfeed.tech/tags/open.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [overlap](<https://devfeed.tech/tags/overlap.md>), [sample](<https://devfeed.tech/tags/sample.md>), [settings](<https://devfeed.tech/tags/settings.md>), [standard](<https://devfeed.tech/tags/standard.md>), [structure](<https://devfeed.tech/tags/structure.md>), [training](<https://devfeed.tech/tags/training.md>)

## AI overview

The paper argues that standard neural collapse can arise on imbalanced datasets when optimization minimizes class-wise averaged losses rather than a global average. It proves this result for cross-entropy and binary cross-entropy losses. In an open-set-inspired setting with classifier vectors for classes lacking training samples, known classes still exhibit standard neural collapse; the unknown-class vectors overlap with one another and are orthogonal to the known-class vectors. The analysis also finds standard neural collapse at the minima of Softmax- and Sigmoid-based contrastive losses rebalanced by inverse sample frequency. These are theoretical results about loss minima.

## Source excerpt

arXiv:2611.07753v1 Announce Type: new Abstract: Neural collapse (NC) describes the elegant geometric structure that sample features and classifier vectors exhibit when a classification model's loss reaches its minimum. Class imbalance, however, generally distorts standard NC. In this paper, we theoretically reformulate NC on imbalanced datasets and in open-set settings. Our analysis is based on an optimization problem over class-wise averaged losses, rather than the global averaged losses used in prior NC studies. We prove that when class-wise averaged losses attain their minima, standard NC is achieved on balanced or imbalanced datasets, for both cross-entropy (CE) and binary cross-entropy (BCE) losses. We further extend the analysis to an open-set-inspired scenario, where some classes contain no training samples yet are still assigned classifier vectors in the losses. We find that in this scenario, when class-wise averaged losses reach their minima, standard NC still emerges on the known classes with training samples, while the classifier vectors of unknown classes without training samples overlap with each other and become orthogonal to those of the known classes. Finally, we analyze Softmax- and Sigmoid-based contrastive losses re-balanced by inverse sample frequency, proving that at their minima on balanced or imbalanced datasets, sample features of different classes likewise form standard NC.