# Accuracy Is Not Risk: Why Medical AI Should Not Report Only Its Most Likely Label

DevFeed: [Accuracy Is Not Risk: Why Medical AI Should Not Report Only Its Most Likely Label](<https://devfeed.tech/articles/accuracy-is-not-risk-why-medical-ai-should-not-report-only-its-most-likely-label-114521.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.01312>)

Author: Jeffrey Humpherys, Makoto Jones

Published: 2026-11-03T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Calibrated uncertainty](<https://devfeed.tech/topics/calibrated-uncertainty.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [AI Models](<https://devfeed.tech/topics/ai-models.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [ai-models](<https://devfeed.tech/tags/ai-models.md>), [cs-cy](<https://devfeed.tech/tags/cs-cy.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [medical](<https://devfeed.tech/tags/medical.md>), [medical-ai](<https://devfeed.tech/tags/medical-ai.md>), [report](<https://devfeed.tech/tags/report.md>), [risk](<https://devfeed.tech/tags/risk.md>), [stat-ap](<https://devfeed.tech/tags/stat-ap.md>), [stat-me](<https://devfeed.tech/tags/stat-me.md>)

## AI overview

A correct most-likely medical AI label can still misstate a patient's downstream outcome risk. The abstract identifies two ways this happens: reporting only the leading label discards probabilities for other states, and assigning the leading diagnosis's risk to a patient who only probably has that diagnosis. Using Jeffrey's rule of conditioning, the authors describe how probabilities over intermediate diagnoses or imaging categories can be carried forward to outcome risk, and when a label and full distribution yield the same decision. Hypothetical chest-pain and BI-RADS examples show how the leading category can understate risk. The authors argue that clinical and insurance decisions should use outcome risk against a threshold justified by the consequences of action and inaction, and that developers should clarify output meaning and check that each workflow step preserves patient risk.

## Source excerpt

arXiv:2611.01312v1 Announce Type: cross Abstract: Background. Many medical AI models are trained to reproduce a single reference label and hand the most likely label to the next step in a clinical workflow. A classifier can select that label correctly while providing an inadequate summary for estimating downstream risk. Methods. We use Jeffrey's rule of conditioning to specify when probabilities over intermediate states, such as diagnoses or imaging categories, can be carried forward to an outcome risk. We separate two errors of reporting only the most likely label, discarding the probability of the other states and applying the risk of patients who have the leading diagnosis to a patient who only probably has it. We also give conditions under which the label and the full distribution lead to the same decision, and we propose guardrails for device developers, regulatory reviewers, and health insurers. Results. In a hypothetical chest-pain differential, reporting only the benign leading diagnosis greatly understates the patient's risk of death or cardiac arrest, so a patient who warrants evaluation could be discharged or denied imaging. In a hypothetical BI-RADS example, the most likely category is "probably benign," yet the propagated malignancy risk exceeds the ceiling that defines that category. Conclusions. A predicted label is not an observed state, and a calibrated classifier is not automatically a valid outcome model. Clinical and insurance coverage decisions should rest on the patient's outcome risk, judged against a threshold justified by the consequences of acting and not acting. Developers should state what each model output means and check that every step between the model and the decision preserves the patient's risk.