# Benchmarks Don't Talk Back: Learning Through Self-Disagreement

DevFeed: [Benchmarks Don't Talk Back: Learning Through Self-Disagreement](<https://devfeed.tech/articles/benchmarks-don-t-talk-back-learning-through-self-disagreement-135080.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.17563>)

Author: Soheil Abbasloo, Peng Cheng

Published: 2026-11-16T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [rlvr](<https://devfeed.tech/topics/rlvr.md>), [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>), [Ground truth / benchmark quality](<https://devfeed.tech/topics/ground-truth-benchmark-quality.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>)

Tags: [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [language-model](<https://devfeed.tech/tags/language-model.md>), [reasoning](<https://devfeed.tech/tags/reasoning.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [training](<https://devfeed.tech/tags/training.md>)

## AI overview

The paper presents dialectical competence--the ability to hold a correct answer against faulty counterarguments while yielding to stronger ones--as distinct from first-answer accuracy. It proposes Self-Disagreement training: a model states a position, generates the strongest opposing case, judges whether it succeeds, and receives reinforcement learning from verifiable rewards. The same reference-based check scores both the answer and the judgment, with no reward model, preference data, human annotation, or second model. The abstract reports that this capability is weak in the base model, can improve without sacrificing first-answer accuracy, and transfers to unseen challengers, including stronger models. It also says accuracy training alone may not improve this competence, and inference-time reflection or additional sampling does not provide it.

## Source excerpt

arXiv:2611.17563v1 Announce Type: cross Abstract: A benchmark usually measures a language model alone with a question, and deployment rarely looks like that: answers get contested, and models give up correct ones under counter-reasoning that is itself wrong. We show that what fails there is not a shortfall of accuracy but a capability in its own right -- knowing when to hold a position and when to yield to a better one -- which we call dialectical competence. It stands on first-answer accuracy without following from it: a model can be made materially better at answering and no better at this, and training for accuracy alone can drift against it. Nor can it be bought at inference, by self-reflection or by sampling more. Taking a cue from the dialectical tradition in philosophy, in which productive opposition arises within a position rather than being imposed on it from outside, we hypothesize that this capability can be trained from opposition the model supplies itself. It states a position, generates the strongest case against that position -- its antithesis -- rules on whether the case stands, and is trained on that ruling, with no second model at training or at deployment. We call this learning through Self-Disagreement: reinforcement learning from verifiable rewards throughout, where the same check that scores an answer against its reference scores the ruling too, so no reward model, no preference data and no human annotation enter anywhere. The capability is trained rather than elicited: weak in the base model, it arrives with the reward, is not paid for out of first-answer accuracy, and holds against challengers the model never trained against, including ones stronger than itself. The opposition a model needs in order to improve, it turns out, need not come from outside it.