# Logically Consistent Accurate Forecasts

DevFeed: [Logically Consistent Accurate Forecasts](<https://devfeed.tech/articles/logically-consistent-accurate-forecasts-135104.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.17735>)

Author: Kate Yuan, Rohan Alur, Manish Raghavan, Devavrat Shah

Published: 2026-11-16T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [Language models](<https://devfeed.tech/topics/language-models.md>)

Tags: [agentic](<https://devfeed.tech/tags/agentic.md>), [brier-score](<https://devfeed.tech/tags/brier-score.md>), [consistency](<https://devfeed.tech/tags/consistency.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [forecasting](<https://devfeed.tech/tags/forecasting.md>), [gpt-5-1](<https://devfeed.tech/tags/gpt-5-1.md>), [large-language-models-llms](<https://devfeed.tech/tags/large-language-models-llms.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [reasoning](<https://devfeed.tech/tags/reasoning.md>), [retrieval](<https://devfeed.tech/tags/retrieval.md>)

## AI overview

The paper addresses contradictions among forecasts for related events, which can violate probability axioms and reduce their usefulness. Rather than simply projecting forecasts onto a consistent set--a strategy the authors say can hurt accuracy--it fine-tunes a supervisory language model to use forecast reasoning to reconcile inconsistencies and improve related predictions together. On ForecastBench, this approach improved the Brier score of a retrieval-augmented GPT-5.1 forecaster by 9.3% and reduced structural inconsistency measures by 82-90%. The abstract reports that tested baselines and ablations did not match both gains simultaneously.

## Source excerpt

arXiv:2611.17735v1 Announce Type: cross Abstract: Agentic forecasting systems based on large language models (LLMs) are now competitive with expert human forecasters, but these systems still suffer from a critical limitation: the inability to produce forecasts that are logically consistent. For example, forecasts for related events often fail to satisfy the axioms of probability, which in turn implies that related forecasts cannot simultaneously be correct and undermines their utility for decision makers. Enforcing consistency is not trivial -- naive approaches (e.g., projection onto the set of consistent forecasts) can degrade accuracy. We argue that the key to overcoming the apparent tradeoff between consistency and accuracy lies in the reasoning which supports each forecast. We finetune a supervisory language model that leverages this reasoning to jointly reconcile inconsistencies while improving the accuracy of related forecasts. On ForecastBench, our method improves the Brier score of a retrieval-augmented GPT-5.1 forecaster by 9.3% while reducing measures of structural inconsistency by 82-90%. We interrogate our method via a comprehensive series of baselines and ablations, and find that no alternative method matches our gains in consistency and accuracy simultaneously.