# Where Is a Prediction Model Miscalibrated, and for Whom? Local Audits in Outcome-Learned Pockets

DevFeed: [Where Is a Prediction Model Miscalibrated, and for Whom? Local Audits in Outcome-Learned Pockets](<https://devfeed.tech/articles/where-is-a-prediction-model-miscalibrated-and-for-whom-local-audits-in-outcome-learned-pockets-114723.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.02639>)

Author: Bernardo Modenesi, Taoyi Chen, Sima Najafzadehkhoei, Lucia Wang, Ameya Diwan, Beatrice Knudsen

Published: 2026-11-03T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Calibrated uncertainty](<https://devfeed.tech/topics/calibrated-uncertainty.md>), [clustering](<https://devfeed.tech/topics/clustering.md>), [generalization in machine learning](<https://devfeed.tech/topics/generalization-in-machine-learning.md>)

Tags: [algorithm](<https://devfeed.tech/tags/algorithm.md>), [apache](<https://devfeed.tech/tags/apache.md>), [audit](<https://devfeed.tech/tags/audit.md>), [audits](<https://devfeed.tech/tags/audits.md>), [black](<https://devfeed.tech/tags/black.md>), [classifiers](<https://devfeed.tech/tags/classifiers.md>), [code](<https://devfeed.tech/tags/code.md>), [confidence](<https://devfeed.tech/tags/confidence.md>), [correlation](<https://devfeed.tech/tags/correlation.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [data](<https://devfeed.tech/tags/data.md>), [error](<https://devfeed.tech/tags/error.md>), [errors](<https://devfeed.tech/tags/errors.md>), [feature](<https://devfeed.tech/tags/feature.md>), [features](<https://devfeed.tech/tags/features.md>), [fixed](<https://devfeed.tech/tags/fixed.md>), [gap](<https://devfeed.tech/tags/gap.md>), [groups](<https://devfeed.tech/tags/groups.md>), [local](<https://devfeed.tech/tags/local.md>), [model](<https://devfeed.tech/tags/model.md>), [prediction](<https://devfeed.tech/tags/prediction.md>), [stat-me](<https://devfeed.tech/tags/stat-me.md>), [stat-ml](<https://devfeed.tech/tags/stat-ml.md>)

## AI overview

The proposed audit finds where prediction errors vary across similar observations and whether demographic groups differ within those pockets, patterns that overall averages and AUC can hide. It forms pockets using random-forest similarity weighted by outcome-predictive feature importance, alongside ridge-regression grouping; learns pockets on one sample and tests them on another with simultaneous confidence intervals that can account for clustering, such as within hospitals. It requires features, outcomes, and fixed predictions. In simulations, the combined audit located planted group-specific errors better than the strongest alternative grouping in 15 of 17 scenarios, with false-alarm rates near or below 5%; results were weaker for local errors unrelated to the outcome. Reported applications found offsetting local errors in care-management scores and chest X-ray classifiers despite no detectable overall error, and localized ICU mortality over-prediction in APACHE IVa.

## Source excerpt

arXiv:2611.02639v1 Announce Type: new Abstract: Overall and group-level averages, and ranking metrics such as the AUC, can hide sets of observations for which a prediction model systematically over- or under-predicts. We present an audit that asks where a model is miscalibrated (in which sets of similar observations it over- or under-predicts on average) and for whom (whether its average prediction error differs between demographic groups, such as by race or sex, within the same set). It groups observations into pockets using random-forest similarity, which weighs each feature by its importance for predicting the outcome, with a ridge-regression grouping alongside. Pockets are learned on one sample and tested on another, with simultaneous confidence intervals that allow for correlation within units such as hospitals; the audit needs only features, outcomes and fixed predictions. In 17 simulation scenarios with an error planted for one group, the combined audit located it better than the strongest alternative grouping in 15, with false-alarm rates near or below 5%; with a local error planted for everyone, it won in 12 of 15 scenarios at the smaller error size and lost when the error was unrelated to the outcome. In public synthetic data from the care-management algorithm studied by Obermeyer et al., it found no detectable overall error but under-prediction of illness in the sickest pockets and over-prediction elsewhere, and distinguished Black patients' concentration where the score under-predicted for everyone from a Black-White gap within pockets. For APACHE IVa, an exploratory analysis located its previously reported over-prediction of ICU mortality in a mostly ventilated pocket and a low-risk pocket, with none detected in a moderate-risk one. Two chest X-ray classifiers showed offsetting local errors behind no detectable overall error. Code: https://github.com/modenesi/pocket-audit