# The Best Second Opinion Is Right Where the Primary Is Wrong: Partner Selection for Selective Span Tagging

DevFeed: [The Best Second Opinion Is Right Where the Primary Is Wrong: Partner Selection for Selective Span Tagging](<https://devfeed.tech/articles/the-best-second-opinion-is-right-where-the-primary-is-wrong-partner-selection-for-selective-span-tagging-108566.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.37678>)

Author: Varun Kotte

Published: 2026-10-30T04:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Uncertainty quantification LLMs](<https://devfeed.tech/topics/uncertainty-quantification-llms.md>), [benchmark overfitting machine learning](<https://devfeed.tech/topics/benchmark-overfitting-machine-learning.md>), [model-deployment](<https://devfeed.tech/topics/model-deployment.md>)

Tags: [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [distribution](<https://devfeed.tech/tags/distribution.md>), [english](<https://devfeed.tech/tags/english.md>), [errors](<https://devfeed.tech/tags/errors.md>), [locale](<https://devfeed.tech/tags/locale.md>), [lora](<https://devfeed.tech/tags/lora.md>), [model](<https://devfeed.tech/tags/model.md>), [selection](<https://devfeed.tech/tags/selection.md>), [zero-shot](<https://devfeed.tech/tags/zero-shot.md>)

## AI overview

For selective span tagging, choose a second model based on how accurately it handles the primary model's errors, rather than its overall span F1. Across 576 pairings on three English benchmarks, this criterion recovered 82% of the best partner's gain, versus 69-70% for span F1 or random selection, and replicated on a fourth dataset. The partner's predictive distribution matters: replacing it with hard labels discarded half its value, while scoring the label actually served improved 94% of pairings. Independently trained partners outperformed readouts of the primary network for LoRA-tuned decoders. In a zero-shot evaluation across 51 MASSIVE locales, each primary had a helpful trained partner, with preferred partners varying by competence.

## Source excerpt

arXiv:2610.37678v1 Announce Type: cross Abstract: A span tagger that ranks its outputs by risk can defer the riskiest ones to a person, and a second model can sharpen this ranking. Which second model should it be? Partner selection is conditional, not global: the best second opinion is the model that is right where the primary is wrong. Across eight tagger types, three seeds and three English benchmarks (576 evaluated primary-partner pairings), choosing the partner by held-out accuracy on the primary's errors recovers 82% of the best partner's gain, against 69-70% for span F1 or a random pick. Frozen before a fourth dataset was chosen, the rule replicated there (Spearman 0.40 vs. 0.04 for span F1). Partner value is also directional. The predictive distribution is the second opinion: hard labels discard half of a partner's value, and scoring the served label improves 94% of pairings. Fixed before the fourth dataset was chosen, this score also replicated there. For LoRA-tuned decoders, independently trained partners beat every readout of the primary network. Applied zero-shot to 51 MASSIVE locales, every primary has a helpful trained partner in every locale, and the preferred partner follows competence.