# Temperature, Not Knowledge: What Recursive Self-Training and Its Remedies Do to a Language Model's Confidence

DevFeed: [Temperature, Not Knowledge: What Recursive Self-Training and Its Remedies Do to a Language Model's Confidence](<https://devfeed.tech/articles/temperature-not-knowledge-what-recursive-self-training-and-its-remedies-do-to-a-language-model-s-confidence-115122.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.01059>)

Author: Dipto Sumit, Syeda Fabiayee Jahan, Farig Sadeque

Published: 2026-11-03T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Uncertainty quantification LLMs](<https://devfeed.tech/topics/uncertainty-quantification-llms.md>), [LLM Techniques](<https://devfeed.tech/topics/llm-techniques.md>), [LLM evaluation / benchmarking](<https://devfeed.tech/topics/llm-evaluation-benchmarking.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>)

Tags: [confidence](<https://devfeed.tech/tags/confidence.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [discrimination](<https://devfeed.tech/tags/discrimination.md>), [distribution](<https://devfeed.tech/tags/distribution.md>), [entropy](<https://devfeed.tech/tags/entropy.md>), [gradient](<https://devfeed.tech/tags/gradient.md>), [knowledge](<https://devfeed.tech/tags/knowledge.md>), [language-model](<https://devfeed.tech/tags/language-model.md>), [qwen3](<https://devfeed.tech/tags/qwen3.md>), [reference](<https://devfeed.tech/tags/reference.md>), [study](<https://devfeed.tech/tags/study.md>), [target](<https://devfeed.tech/tags/target.md>), [temperature](<https://devfeed.tech/tags/temperature.md>), [top](<https://devfeed.tech/tags/top.md>), [top-p](<https://devfeed.tech/tags/top-p.md>), [training](<https://devfeed.tech/tags/training.md>)

## AI overview

In a five-round self-training study on Qwen3-1.7B, accuracy and overall discrimination stayed nearly unchanged while confidence sharpened and the most confident answers became less reliable. Most confidence drift came from fitting each generated answer twice, with top-p truncation adding to it; removing both kept entropy on generated outputs flat. Matching predictive entropy to the calibrated starting model each round recalibrated nearly as well as label-based temperature fitting, including at 8B. Remedies differed: truncated cross-entropy, an entropy bonus, and single-pass training preserved confidence temperature without improving discrimination; focal loss and multi-sample targets improved discrimination but led to underconfidence, while reference-data mixing overshot. Combining a ranking-improving loss with label-free recalibration achieved both goals.

## Source excerpt

arXiv:2611.01059v1 Announce Type: new Abstract: Recursive self-training can collapse language models, and many proposed remedies target overconfidence, yet their effects on confidence are rarely measured. We study a five-round self-training loop on Qwen3-1.7B, using three seeds per compared method, and distinguish confidence temperature (drift from a calibrated starting point) from discrimination (ranking correct answers above incorrect ones). Accuracy and overall discrimination remain nearly unchanged, but confidence sharpens and the most confident answers become less reliable. Matching predictive entropy to the calibrated starting model each round recalibrates nearly as well as label-based temperature fitting, at both 1.7B and 8B. Sharpening is not inherent to self-training: exact samples from a model's distribution have zero expected gradient. Instead, most drift comes from fitting each generated answer twice, with top-p truncation contributing the rest; removing both keeps entropy on generated outputs flat. Remedies affect the two properties differently: truncated cross-entropy, an entropy bonus, and single-pass training preserve temperature without improving discrimination, while focal loss and multi-sample targets improve discrimination but cause underconfidence. Reference-data mixing also overshoots. Pairing a ranking-improving loss with label-free recalibration achieves both.