# Selective Classification Can Magnify Disparities Across Groups

DevFeed: [Selective Classification Can Magnify Disparities Across Groups](<https://devfeed.tech/articles/selective-classification-can-magnify-disparities-across-groups-7589.md>)

Original publisher: [Read original article](<https://ai.stanford.edu/blog/sc-magnifies-disparities/>)

Author: A Href; Erik Jones

Published: 2021-10-13T07:00:00Z

Content type: article

Language: en

Sources: [The Stanford AI Lab Blog](<https://devfeed.tech/sources/the-stanford-ai-lab-blog.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [model-deployment](<https://devfeed.tech/topics/model-deployment.md>), [data](<https://devfeed.tech/topics/data.md>), [datasets](<https://devfeed.tech/topics/datasets.md>), [Natural language processing](<https://devfeed.tech/topics/nlp.md>)

Tags: [iclr](<https://devfeed.tech/tags/iclr.md>), [ml](<https://devfeed.tech/tags/ml.md>), [model](<https://devfeed.tech/tags/model.md>), [models](<https://devfeed.tech/tags/models.md>), [nlp](<https://devfeed.tech/tags/nlp.md>), [overview](<https://devfeed.tech/tags/overview.md>), [vision](<https://devfeed.tech/tags/vision.md>)

## AI overview

The article examines selective classification, in which machine-learning models can abstain when uncertain. It shows that although abstention can improve average accuracy, it may fail to help or may worsen accuracy for clinically important subgroups, such as patients with pleural effusion who have not yet received a chest drain. The article discusses the underlying failure mode, theoretical implications, and approaches for building more equitable selective classifiers.

## Source excerpt

Selective classification, where models are allowed to "abstain" when they are uncertain about a prediction, is a useful approach for deploying models in settings where errors are costly. For example, in medicine, model errors can have life-or-death ramifications, but abstentions can be easily handled by backing off to a doctor, who then makes a diagnosis. Across a range of applications from vision 123 and NLP 45, even simple selective classifiers, relying only on model logits, routinely and often dramatically improve accuracy by abstaining. This makes selective classification a compelling tool for ML practitioners 67. However, in our recent ICLR paper, we find that despite reliably improving average accuracy, selective classification can fail to improve and even hurt the accuracy over certain subpopulations of the data. As a motivating example, consider the task of diagnosing pleural effusion, or fluid in the lungs, from chest X-rays. Pleural effusion is often treated with a chest drain, so many pleural effusion cases also have chest drains, while most cases without pleural effusion do not have chest drains 8. While selective classification improves average accuracy for this task, we find that it does not appreciably improve accuracy on the most clinically relevant subgroup, or subpopulation, of the data: those that have pleural effusion but don't yet have a chest drain, i.e. those that have pleural effusion but have not yet been treated for it. Practitioners, thus, should be wary of these potential failure modes of using selective classification in the wild. Example of the spurious correlation setup. This patient has a pleural effusion (excess fluid in the lung), but does not yet have a chest drain. The model, relying on the presence of a chest drain to make a prediction, incorrectly predicts negative. To further outline this critical failure mode of selective classification, we'll first provide an overview of selective classification. We then demonstrate empirical