# Tabular foundation models are not a shortcut to calibrated clinical risk prediction: a pre-registered simulation against the true risk

DevFeed: [Tabular foundation models are not a shortcut to calibrated clinical risk prediction: a pre-registered simulation against the true risk](<https://devfeed.tech/articles/tabular-foundation-models-are-not-a-shortcut-to-calibrated-clinical-risk-prediction-a-pre-registered-simulation-against-the-true-risk-108763.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.39065>)

Author: Ebrahim Khaled Ebrahim

Published: 2026-10-30T04:00:00Z

Content type: news

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [AI Foundation Models](<https://devfeed.tech/topics/ai-foundation-models.md>), [Calibrated uncertainty](<https://devfeed.tech/topics/calibrated-uncertainty.md>)

Tags: [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [foundation-models](<https://devfeed.tech/tags/foundation-models.md>), [models](<https://devfeed.tech/tags/models.md>), [risk](<https://devfeed.tech/tags/risk.md>), [simulation](<https://devfeed.tech/tags/simulation.md>), [stat-me](<https://devfeed.tech/tags/stat-me.md>)

## AI overview

In a pre-registered simulation with known true risks, TabPFN-3.5, TabICLv2 and Mitra-v2 did not consistently improve calibration over Firth logistic regression with intercept correction (FLIC). Calibration slopes were similar in 36-37 of 48 core scenarios, and foundation models were better only at n=200 with a 3% event rate. At n=3,000, risks were often close to truth for FLIC, TabPFN-3.5 and Mitra-v2, but much less often for TabICLv2, especially at a 3% event rate. Across random designs, foundation models had lower true calibration error than FLIC beyond the stated margin in 25%-38% of cases and higher error in none, though small losses were predicted at n=3,000. In four clinical data sets, no model resisted population changes in event rate; recalibration using 500 local patients largely repaired the shift.

## Source excerpt

arXiv:2610.39065v1 Announce Type: cross Abstract: Tabular foundation models are pretrained networks that predict risk from a table without fitting a model. Whether their risks are correct (calibrated) has been judged on real data, where true risk is unknown. In a pre-registered simulation with known true risks, we compared TabPFN-3.5, TabICLv2 and Mitra-v2 with Firth logistic regression with intercept correction (FLIC) in 58 scenarios and 1,000 random designs, then on four clinical data sets. Foundation models were not a shortcut to calibration. Their calibration slope matched FLIC's within the pre-registered margin in 36 or 37 of 48 core scenarios and was better only at n = 200 with a 3% event rate. Even 3,000 patients were often not enough: with FLIC, TabPFN-3.5 or Mitra-v2, 63%-83% of samples gave risks close to the truth in average, intercept and slope at event rates of 10% and 30%, and 35%-50% at 3%. TabICLv2 did worse (8%-39% and 1%-3%). Across the random designs, foundation models had a lower true calibration error than FLIC beyond the margin in 25%-38% and a higher one in none, though small losses were predicted at n = 3000. On real data, no model resisted a change in event rate between populations; recalibration on 500 local patients largely repaired it.