# CacheGraft: Enhancing Retrieval-Augmented Generation via Latent-Space Document Enrichment

DevFeed: [CacheGraft: Enhancing Retrieval-Augmented Generation via Latent-Space Document Enrichment](<https://devfeed.tech/articles/cachegraft-enhancing-retrieval-augmented-generation-via-latent-space-document-enrichment-120609.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.07393>)

Author: Vihan Lakshman, Peter Baile Chen, Zhuohan Gu, Michael Cafarella

Published: 2026-11-06T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Retrieval Augmented Generation (RAG)](<https://devfeed.tech/topics/retrieval-augmented-generation-rag.md>), [Enhancing AI model performance](<https://devfeed.tech/topics/enhancing-ai-model-performance.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [search-engines](<https://devfeed.tech/topics/search-engines.md>)

Tags: [agent-communication](<https://devfeed.tech/tags/agent-communication.md>), [agents](<https://devfeed.tech/tags/agents.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [cs-ir](<https://devfeed.tech/tags/cs-ir.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [enrichment](<https://devfeed.tech/tags/enrichment.md>), [framework](<https://devfeed.tech/tags/framework.md>), [fusion](<https://devfeed.tech/tags/fusion.md>), [generation](<https://devfeed.tech/tags/generation.md>), [inference](<https://devfeed.tech/tags/inference.md>), [kv-cache](<https://devfeed.tech/tags/kv-cache.md>), [language-model](<https://devfeed.tech/tags/language-model.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [model](<https://devfeed.tech/tags/model.md>), [offline](<https://devfeed.tech/tags/offline.md>), [performance](<https://devfeed.tech/tags/performance.md>), [qa](<https://devfeed.tech/tags/qa.md>), [rag](<https://devfeed.tech/tags/rag.md>), [rag-systems](<https://devfeed.tech/tags/rag-systems.md>), [receiver](<https://devfeed.tech/tags/receiver.md>), [retrieval](<https://devfeed.tech/tags/retrieval.md>), [retrieval-augmented-generation](<https://devfeed.tech/tags/retrieval-augmented-generation.md>)

## AI overview

CacheGraft enriches retrieval-augmented generation by encoding corpus passages offline as fused key-value cache representations between a 4B-parameter sharer model and a 0.6B-parameter receiver. At query time, the receiver can use the cache without deploying the larger model. Across five QA benchmarks and two locally run judges, it improved answer quality over a text-only receiver in 29 of 30 comparisons; 17 improvements were statistically significant. The reported gains were zero-shot for the fuser, which was not trained on retrieval data. These initial results suggest potential for settings where inference must use a small model.

## Source excerpt

arXiv:2611.07393v1 Announce Type: cross Abstract: Recent work in machine learning has demonstrated that transformer-based language model agents can communicate state not only through text, but also directly in latent space via key-value (KV) caches. In this paper, we argue that such cache-to-cache translation can provide utility beyond the original motivating application of real-time agent communication and can instead be utilized for retrieval-augmented generation (RAG). Building on the Cache-to-Cache (C2C) fusion framework, our proposed system CacheGraft encodes each corpus passage offline into a fused KV cache representation between a larger sharer model (4B parameters in our experiments) and a smaller receiver model (0.6B parameters) such that this cache can be injected into the receiver at query time with no need for the larger model. Across five QA benchmarks and two independent locally-run judges, CacheGraft improves answer quality over the text-only receiver baseline in 29 out of 30 comparisons, of which 17 of the 30 are statistically significant. Notably, these gains are zero-shot with respect to the fuser, which was never trained on retrieval data. These initial results suggest that latent-space document enrichment may be a promising avenue for enhancing the performance of RAG systems, particularly in compute-constrained settings where it is only feasible to deploy a small model at inference time.