# LLMs Are Not (Consistently) Bayesian: Quantifying Internal (In)consistencies of LLMs' Probabilistic Beliefs

DevFeed: [LLMs Are Not (Consistently) Bayesian: Quantifying Internal (In)consistencies of LLMs' Probabilistic Beliefs](<https://devfeed.tech/articles/llms-are-not-consistently-bayesian-quantifying-internal-in-consistencies-of-llms-probabilistic-beliefs-6730.md>)

Original publisher: [Read original article](<https://machinelearning.apple.com/research/llms-not-consistently-bayesian>)

Published: 2026-08-28T00:00:00Z

Content type: article

Language: en

Sources: [Apple Machine Learning Research](<https://devfeed.tech/sources/apple-machine-learning-research.md>)

Topics: [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Uncertainty quantification LLMs](<https://devfeed.tech/topics/uncertainty-quantification-llms.md>), [AI Inference](<https://devfeed.tech/topics/ai-inference.md>), [Data Science](<https://devfeed.tech/topics/data-science.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [data-science](<https://devfeed.tech/tags/data-science.md>), [diagnostics](<https://devfeed.tech/tags/diagnostics.md>), [llms](<https://devfeed.tech/tags/llms.md>), [research](<https://devfeed.tech/tags/research.md>)

## AI overview

This research examines whether large language models update probabilistic beliefs in accordance with Bayes' rule. It introduces the information processing gap to quantify deviations from Bayesian updates, compares evidence-integration approaches, and finds that heuristic, non-Bayesian updates can outperform exact Bayesian updates on downstream tasks.

## Source excerpt

Modern AI systems are being deployed in complex domains such as medicine, science, and law, where there is often not a single correct answer given the observed evidence. Such systems must be able to represent and update uncertain beliefs about the world as new evidence arrives to make rational decisions. We introduce the novel technique of studying LLMs as information processing rules and utilize the information processing gap--the deviation from Bayes updates--to study the internal (in)consistencies of how LLMs update their probabilistic beliefs from evidence. Our extensive experiments evaluate...