# RECAP: Context-Aware Prediction Recycling with Request-Local and Persistent Memories for Multimodal Speculative Decoding

DevFeed: [RECAP: Context-Aware Prediction Recycling with Request-Local and Persistent Memories for Multimodal Speculative Decoding](<https://devfeed.tech/articles/recap-context-aware-prediction-recycling-with-request-local-and-persistent-memories-for-multimodal-speculative-decoding-118396.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.05894>)

Author: Hengjie Zhu, Dayan Wu, Xinze Liu, Peng Fu, Zheng Lin, Weiping Wang

Published: 2026-11-05T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Cache](<https://devfeed.tech/topics/cache.md>)

Tags: [autoregressive](<https://devfeed.tech/tags/autoregressive.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [context](<https://devfeed.tech/tags/context.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [local](<https://devfeed.tech/tags/local.md>), [memory](<https://devfeed.tech/tags/memory.md>), [multimodal](<https://devfeed.tech/tags/multimodal.md>), [predictions](<https://devfeed.tech/tags/predictions.md>), [recap](<https://devfeed.tech/tags/recap.md>), [recycling](<https://devfeed.tech/tags/recycling.md>), [request](<https://devfeed.tech/tags/request.md>), [reuse](<https://devfeed.tech/tags/reuse.md>), [speculative-decoding](<https://devfeed.tech/tags/speculative-decoding.md>), [speed](<https://devfeed.tech/tags/speed.md>), [target](<https://devfeed.tech/tags/target.md>), [vision-language-models](<https://devfeed.tech/tags/vision-language-models.md>)

## AI overview

RECAP reuses a vision-language model's own predictions for speculative decoding while reducing errors caused by matching only the last token across different images and prefixes. It combines one-, two-, and three-token keys: contextual memories are limited to the current request, while token-level memory persists across requests. Entropy-confidence weighting fuses retrieved predictions, and best-first search builds a candidate tree that one target-model pass verifies under greedy decoding. The method requires neither training nor an auxiliary draft model. Across seven single-turn benchmarks, it reports mean speedups of 1.66x on Qwen2.5-VL-7B and 1.56x on LLaVA-1.6-7B, compared with 1.51x and 1.37x for Token Recycling. These are abstract-level results; the supplied text does not provide benchmark details.

## Source excerpt

arXiv:2611.05894v1 Announce Type: new Abstract: Recycling a target model's own predictions offers a training-free source of proposals for speculative decoding. In vision-language models, however, existing recycling indexes predictions by the last token alone, conflating predictions computed under different images and prefixes; we find that such reuse fails most often at image-sensitive decoding states. Longer keys reduce this aliasing but match fewer states. We introduce \method, which stores target predictions under one-, two-, and three-token keys: contextual memories live only for the current request and are thus implicitly conditioned on its image, while token-level memory persists across requests to retain coverage. At each lookup, entropy-confidence weighting fuses the retrieved predictions according to their key specificity and concentration. Best-first search composes the fused transitions into a candidate tree, supplemented by suffix-matched continuations, and one target pass verifies the tree under greedy decoding. Without training or an auxiliary draft model, \method\ achieves mean speedups over autoregressive decoding of $1.66\speed$ and $1.56\speed$ across seven single-turn benchmarks on Qwen2.5-VL-7B and LLaVA-1.6-7B, exceeding the strongest training-free baseline, Token Recycling ($1.51\speed$ and $1.37\speed$). Our implementation is available in https://github.com/Senmo996/RECAP-official-code.