# Comparing Classification and Feature Learning by Analyzing Their Loss Functions in Normalized Spaces with Different Scale Factors

DevFeed: [Comparing Classification and Feature Learning by Analyzing Their Loss Functions in Normalized Spaces with Different Scale Factors](<https://devfeed.tech/articles/comparing-classification-and-feature-learning-by-analyzing-their-loss-functions-in-normalized-spaces-with-different-scale-factors-120658.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.07741>)

Author: Qiufu Li, Huibin Xiao, Linlin Shen, Weijia Fan, Xi Jia, Yongdong Huang

Published: 2026-11-06T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Representation learning](<https://devfeed.tech/topics/representation-learning.md>), [Neural Network](<https://devfeed.tech/topics/neural-network.md>)

Tags: [analyze](<https://devfeed.tech/tags/analyze.md>), [behaviors](<https://devfeed.tech/tags/behaviors.md>), [classification](<https://devfeed.tech/tags/classification.md>), [classification-accuracy](<https://devfeed.tech/tags/classification-accuracy.md>), [convergence](<https://devfeed.tech/tags/convergence.md>), [cross](<https://devfeed.tech/tags/cross.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [entropy](<https://devfeed.tech/tags/entropy.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [factor](<https://devfeed.tech/tags/factor.md>), [feature](<https://devfeed.tech/tags/feature.md>), [features](<https://devfeed.tech/tags/features.md>), [gamma](<https://devfeed.tech/tags/gamma.md>), [global](<https://devfeed.tech/tags/global.md>), [large](<https://devfeed.tech/tags/large.md>), [lead](<https://devfeed.tech/tags/lead.md>), [learning](<https://devfeed.tech/tags/learning.md>), [local](<https://devfeed.tech/tags/local.md>), [loss-functions](<https://devfeed.tech/tags/loss-functions.md>), [scale](<https://devfeed.tech/tags/scale.md>), [spaces](<https://devfeed.tech/tags/spaces.md>)

## AI overview

The paper argues that classification accuracy and feature quality capture different properties: accuracy is sample-wise, while feature learning seeks dataset-level compactness within classes and separation between classes. It redesigns binary cross-entropy (BCE) as a counterpart to cross-entropy (CE), and reports that minima of both losses in normalized feature space can produce neural collapse. With a fixed training strategy, a very large scale factor γ can yield perfect accuracy but slows convergence toward ideal feature geometry; a very small γ makes classification sensitive to small perturbations. The authors say a moderate γ is needed to balance reliable classification and well-structured features. Experiments reported in the abstract show BCE advantages over CE in long-tail and open-set recognition, attributed to better alignment with feature-learning goals.

## Source excerpt

arXiv:2611.07741v1 Announce Type: new Abstract: Classification and feature learning are two closely interrelated fundamental tasks. Feature learning in supervised settings is typically achieved by training classification models, under the common assumption that higher classification accuracy implies better feature quality. In this paper, we first point out that classification accuracy reflects the sample-wise local properties of a model on a dataset, whereas feature learning aims to learn features with good dataset-level global properties, such as intra-class compactness and inter-class separability. From the first principle of feature learning, we redesign the binary cross-entropy (BCE) loss, a counterpart of the cross-entropy (CE) loss used in classification. Then, in normalized feature space, we analyze the minima of the BCE and CE losses and show that both can lead to neural collapse, where sample features from different classes become maximally compact and maximally separated. More importantly, by investigating the convergence behaviors of the two losses as the scale factor $\gamma$ varies in the normalized space, we reveal how classification accuracy and feature quality change differently across spaces of different sizes. Under a fixed training strategy, a very large $\gamma$ achieves perfect classification accuracy but slows convergence toward the ideal feature geometry, whereas a very small $\gamma$ makes classification highly sensitive to small perturbations. A moderate $\gamma$ is therefore necessary for simultaneously achieving reliable classification and well-structured features. Extensive experiments validate these findings and demonstrate that BCE loss exhibits advantages in long-tail and open-set recognitions, as it better aligns with the global requirements of feature learning.