# Contextual Embeddings Beyond the Gold Chunk

DevFeed: [Contextual Embeddings Beyond the Gold Chunk](<https://devfeed.tech/articles/contextual-embeddings-beyond-the-gold-chunk-108734.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.38854>)

Author: Sedigheh Eslami, Markus Krimmel, William Barber, Antoine Chaffin, Maximilian Schall, Jonas Golde, Louis Milliken, Ekaterina Glazkova, Denis Bykov, Bo Wang

Published: 2026-10-30T04:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Embeddings](<https://devfeed.tech/topics/embeddings.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [LLM evaluation / benchmarking](<https://devfeed.tech/topics/llm-evaluation-benchmarking.md>), [vlm](<https://devfeed.tech/topics/vlm.md>)

Tags: [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [context-compression](<https://devfeed.tech/tags/context-compression.md>), [contextual-embeddings](<https://devfeed.tech/tags/contextual-embeddings.md>), [cs-cl](<https://devfeed.tech/tags/cs-cl.md>), [cs-ir](<https://devfeed.tech/tags/cs-ir.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [embeddings](<https://devfeed.tech/tags/embeddings.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [image](<https://devfeed.tech/tags/image.md>), [leaderboard](<https://devfeed.tech/tags/leaderboard.md>), [relevance](<https://devfeed.tech/tags/relevance.md>), [retrieval](<https://devfeed.tech/tags/retrieval.md>), [search](<https://devfeed.tech/tags/search.md>), [training](<https://devfeed.tech/tags/training.md>)

## AI overview

The paper introduces pplx-embed-v2-context, contextual embedding models designed to retrieve multiple answer-bearing chunks and the evidence needed to verify answers. They are trained by distilling graded chunk relevance from a query-aware context compression model, avoiding gold-chunk labels and making training targets independent of chunking. The released 9B and 0.6B models use compatible embedding spaces. The authors introduce Context-bench, a controlled diagnostic, and Q2D-Chunk, based on real-world search queries, which is also published as a leaderboard. They report that the 9B model sets state-of-the-art results on both benchmarks and ConTEB. The models also achieve state-of-the-art performance among dense retrievers on visual ViDoRe-V3, with gains from granular image chunking.

## Source excerpt

arXiv:2610.38854v1 Announce Type: new Abstract: Contextual embeddings represent chunks of a document with respect to the document's global context. These models are typically trained and evaluated under the assumption that a relevant document contains only a single relevant chunk. In this work, we argue that in real-world use cases a retriever should not only find answers spread over several chunks but also surface the evidence needed to verify them. To address this, we introduce pplx-embed-v2-context, a family of contextual embedding models trained by distilling graded chunk-level relevance from a query-aware context compression model. This removes the need for gold-chunk labels and makes the training targets independent of the chunking. We utilize a vision-language backbone for training and release pplx-embed-v2-context in 9B and 0.6B sizes with compatible embedding spaces. On the evaluation side, we introduce two benchmarks that label both the chunks that answer a query and the evidence needed to verify the answer. Context-bench is a controlled diagnostic, and Q2D-Chunk is built from real-world search queries. To encourage further research, we also publish Q2D-Chunk as a public leaderboard. Our 9B model sets a new state of the art on both benchmarks and ConTEB. We find that the visual retrieval capabilities of our 9B model transfer from its vision-language base model and propose to extend chunking to image inputs. We demonstrate that our models achieve state-of-the-art performance among dense retrievers on visual ViDoRe-V3 and that performance improves with granular image chunking.