# ThinkJev: One Thought, Every Budget for Calibrated Typed Decisions

DevFeed: [ThinkJev: One Thought, Every Budget for Calibrated Typed Decisions](<https://devfeed.tech/articles/thinkjev-one-thought-every-budget-for-calibrated-typed-decisions-107959.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.37732>)

Author: Xiaolu Ding, Kai Yang, Dongdong Liu

Published: 2026-10-30T04:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [sequential-testing](<https://devfeed.tech/topics/sequential-testing.md>), [Benchmark](<https://devfeed.tech/topics/benchmark.md>)

Tags: [2](<https://devfeed.tech/tags/2.md>), [abstract](<https://devfeed.tech/tags/abstract.md>), [arithmetic](<https://devfeed.tech/tags/arithmetic.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [choice](<https://devfeed.tech/tags/choice.md>), [confidence](<https://devfeed.tech/tags/confidence.md>), [costs](<https://devfeed.tech/tags/costs.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [error](<https://devfeed.tech/tags/error.md>), [guarantees](<https://devfeed.tech/tags/guarantees.md>), [has](<https://devfeed.tech/tags/has.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [qwen3-5](<https://devfeed.tech/tags/qwen3-5.md>), [thinking](<https://devfeed.tech/tags/thinking.md>), [tokens](<https://devfeed.tech/tags/tokens.md>)

## AI overview

ThinkJev combines direct and reasoning-based typed decisions from one hybrid-thinking model, reading probabilities over the same answer options in both modes. Escalation reuses the cached prompt, and a single sampled reasoning trace can be evaluated at different token budgets. Without training, a statistical test on labeled development data limits the fast system's accepted answers to a target error rate: at a 5% target, Qwen3.5-27B handled 24.5% of 8,359 test decisions directly with 3.9% realized error. For remaining decisions, stopping reasoning when the answer becomes confident achieved 81.6% accuracy using 72% of the thought tokens required by always thinking (82.7% accuracy). Confidence-based selection of which decisions to think through saved less. In an exploratory JevBench comparison, Qwen3.5-27B thought on 29.4% of public items and its accuracy was not distinguishable from the two listed thinking APIs.

## Source excerpt

arXiv:2610.37732v1 Announce Type: new Abstract: Many uses of language models end in a typed decision: a choice among a closed set of options, whose probabilities code can act on. Reading those probabilities from one forward pass is fast but often wrong on decisions that take several steps, such as arithmetic, counting, and date comparison. Letting the model think first fixes most of these errors, but costs seconds per decision instead of milliseconds. To pay for thinking only where it helps, ThinkJev builds a fast and a slow system from one open hybrid-thinking model that can answer directly or after a thought. System 1 reads the model's probabilities for the option letters when it answers directly; System 2 reads the same letters after the model has thought on the same cached prompt, so escalating repeats no work and both systems give probabilities over the same options. Because the letters can be read after any number of thought tokens, one sampled thought also gives the answer under every thinking budget. Without training, ThinkJev lets System 1 answer alone only where a statistical test on labeled dev data guarantees that its accepted answers stay under a target error rate. At a 5% target, Qwen3.5-27B answers 24.5% of 8359 test decisions this way with a realized error of 3.9% (Qwen3.5-4B and 9B: 6.5% and 0.0%). For the rest, stopping each thought as soon as its answer is confident reaches 81.6% accuracy, against 82.7% for always thinking, with 72% of the thought tokens. Choosing when to think from System 1's confidence alone saves less, because the decisions System 1 is least sure of are often not the ones thinking fixes. On the public items of JevBench, an independent benchmark of typed-decision services, Qwen3.5-27B thinks on 29.4% and is not distinguishable in accuracy from the two thinking APIs listed (exploratory). Code and data generators are available at https://github.com/ding-xiaolu/ThinkJev.