# FondueQA: A Benchmark for Evaluating Parametric Compensation in Multi-Hop QA

DevFeed: [FondueQA: A Benchmark for Evaluating Parametric Compensation in Multi-Hop QA](<https://devfeed.tech/articles/fondueqa-a-benchmark-for-evaluating-parametric-compensation-in-multi-hop-qa-123387.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.09274>)

Author: Laurin Schmid, Suman Navaratnarajah, Koel Dutta Chowdhury, Steffen Eger, Josif Grabocka, Christian Frey

Published: 2026-11-09T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [LLM evaluation / benchmarking](<https://devfeed.tech/topics/llm-evaluation-benchmarking.md>), [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [Retrieval Augmented Generation (RAG)](<https://devfeed.tech/topics/retrieval-augmented-generation-rag.md>), [benchmark overfitting machine learning](<https://devfeed.tech/topics/benchmark-overfitting-machine-learning.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [based](<https://devfeed.tech/tags/based.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [bridging](<https://devfeed.tech/tags/bridging.md>), [but](<https://devfeed.tech/tags/but.md>), [can](<https://devfeed.tech/tags/can.md>), [capability](<https://devfeed.tech/tags/capability.md>), [context](<https://devfeed.tech/tags/context.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-ir](<https://devfeed.tech/tags/cs-ir.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [either](<https://devfeed.tech/tags/either.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [fail](<https://devfeed.tech/tags/fail.md>), [general](<https://devfeed.tech/tags/general.md>), [isolation](<https://devfeed.tech/tags/isolation.md>), [knowledge](<https://devfeed.tech/tags/knowledge.md>), [language](<https://devfeed.tech/tags/language.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [large](<https://devfeed.tech/tags/large.md>), [qa](<https://devfeed.tech/tags/qa.md>)

## AI overview

FondueQA tests how language models use internal knowledge when retrieval evidence is incomplete, while reducing ambiguity from benchmark-specific memorization: each required multi-hop fact is uniquely assigned to a designated evidence document, which can be ablated in a controlled way. The reported results show that compensation varies with entity popularity and the role of the missing evidence. Models generally compensate better for facts about popular entities, and bridging evidence can help more than the answer-bearing document. A model may recall a missing fact in isolation yet fail to apply it when partial context includes plausible competing entities, reducing Exact Match by up to 15 percentage points.

## Source excerpt

arXiv:2611.09274v1 Announce Type: new Abstract: Large Language Models (LLMs) show remarkable performance based on parametric knowledge, i.e., their ability to recall facts and associations internalized in a model's weights. However, studying this capability on established multi-hop benchmarks may be complicated by benchmark contamination, as many of these benchmarks have been publicly available long enough to plausibly appear in modern pretraining corpora. As a result, correct answers under missing evidence may reflect either general parametric knowledge or memorization of benchmark-specific instances. To study how models use internal knowledge when retrieval evidence is incomplete, we introduce FondueQA, in which every required multi-hop fact is uniquely localized to a designated evidence document. This enables controlled document-level evidence ablations to study parametric compensation under incomplete retrieval. Our empirical evaluation reveals that parametric compensation strongly depends on both entity popularity and the structural role of missing evidence. Generally, models compensate more successfully for facts involving popular entities. On FondueQA, we show that bridging evidence can be more useful than the answer-bearing document itself. Moreover, models may recall a missing fact in isolation but fail to use it under partial evidence, particularly when the context contains plausible competing entities, which can reduce Exact Match by up to 15 percentage points.