# A Contradiction Judge Counted Reported Teammate Errors as Hallucinations: An Assertion-Aware Re-Score of One Multi-Agent LLM Harness

DevFeed: [A Contradiction Judge Counted Reported Teammate Errors as Hallucinations: An Assertion-Aware Re-Score of One Multi-Agent LLM Harness](<https://devfeed.tech/articles/a-contradiction-judge-counted-reported-teammate-errors-as-hallucinations-an-assertion-aware-re-score-of-one-multi-agent-llm-harness-131939.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2606.21666>)

Author: Carson Rodrigues

Published: 2026-11-13T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Multi Agent Systems](<https://devfeed.tech/topics/multi-agent-systems.md>), [AI agent hallucinations](<https://devfeed.tech/topics/ai-agent-hallucinations.md>), [Agent Harness](<https://devfeed.tech/topics/agent-harness.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-ma](<https://devfeed.tech/tags/cs-ma.md>), [harness](<https://devfeed.tech/tags/harness.md>), [llm](<https://devfeed.tech/tags/llm.md>), [multi-agent](<https://devfeed.tech/tags/multi-agent.md>)

## AI overview

The study re-scores saved outputs from one multi-agent LLM harness, separating a model's own incorrect assertions from reports attributed to teammates. Across 1,832 outputs, three LLM judges labeled six facts per output. For values asserted as the model's own, the broadcast effect was −0.0208 (95% CI −0.0458 to +0.0041) over 10 Full-Broadcast trials, not distinguishable from zero; equivalence to zero holds only within a post hoc margin and under the specified prompt. Counting mentions instead reproduced a +0.1996 effect, with teammate-error reports accounting for nearly all of the Haiku travel effect in these judge labels. The paper withdraws the earlier preprint's contamination claim; its runs cannot be re-scored. The revised analysis covers one harness and 13 hand-written scenarios, and its labels lack human validation, so the reported shares may not transfer.

## Source excerpt

arXiv:2606.21666v2 Announce Type: replace Abstract: LLM judges that score hallucination flag any statement that contradicts a reference fact. Agents in multi-agent LLM systems quote and dispute each other's claims, so such a judge can count a correct report of a teammate's error as a hallucination. An earlier version of this study, a preprint (arXiv:2606.21666v1), reported that broadcasting raised a hallucination rate from 0.492 to 0.658 (+34%) and called it contamination; this version withdraws that claim, and the preprint's runs cannot be re-scored. An unpublished revision of that harness reported 0.289 to 0.554 (+0.265, 95% CI [0.225, 0.305]), and its saved outputs are re-scored here. Under an analysis plan fixed before any judge call, three judges from three vendors labeled each of six facts in 1,832 outputs as asserted wrong, attributed to another source, asserted correct or absent. On values that outputs assert as their own, the broadcast effect was -0.0208 (95% CI [-0.0458, +0.0041]) on 10 Full-Broadcast trials: not distinguishable from zero, and equivalent to zero only within a post hoc margin of +/-0.0663 and under the plan's prompt. On these LLM-judge labels, reports of a teammate's wrong value accounted for nearly all of the Haiku travel effect. Scored for mentions, the judges reproduced the effect (+0.1996). With Llama agents, 2 of 40 teammate outputs asserted the wrong destination. One of twelve Claude judge-prompt combinations gave an asserted increase (Sonnet 4.6, one paraphrase: +0.0246, 95% CI [+0.0055, +0.0436]). In 40 new trials per arm, judged by Claude models like the agents, the asserted difference was -0.0505 when both judges agreed and +0.0161 (95% CI [+0.0024, +0.0299]) when either did. Main limit: the labels have no human validation; human labels for 120 outputs are pending. The outputs come from one harness and 13 hand-written scenarios, so the shares do not transfer.