# FOCUS: Closed-Loop Attention Feedback for Efficient Vision-Language Understanding

DevFeed: [FOCUS: Closed-Loop Attention Feedback for Efficient Vision-Language Understanding](<https://devfeed.tech/articles/focus-closed-loop-attention-feedback-for-efficient-vision-language-understanding-115163.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.01438>)

Author: Akul Santhosh, Aastha Jhunjhunwala

Published: 2026-11-03T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [vlm](<https://devfeed.tech/topics/vlm.md>), [Machine Perception](<https://devfeed.tech/topics/machine-perception.md>), [LLM optimization](<https://devfeed.tech/topics/llm-optimization.md>)

Tags: [attention](<https://devfeed.tech/tags/attention.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [embedding](<https://devfeed.tech/tags/embedding.md>), [feedback](<https://devfeed.tech/tags/feedback.md>), [focus](<https://devfeed.tech/tags/focus.md>), [inference](<https://devfeed.tech/tags/inference.md>), [motion](<https://devfeed.tech/tags/motion.md>), [patch](<https://devfeed.tech/tags/patch.md>), [relevance](<https://devfeed.tech/tags/relevance.md>), [sampling](<https://devfeed.tech/tags/sampling.md>), [semantic](<https://devfeed.tech/tags/semantic.md>), [spatial](<https://devfeed.tech/tags/spatial.md>), [streaming](<https://devfeed.tech/tags/streaming.md>), [temporal](<https://devfeed.tech/tags/temporal.md>), [text](<https://devfeed.tech/tags/text.md>), [token](<https://devfeed.tech/tags/token.md>), [tokens](<https://devfeed.tech/tags/tokens.md>), [training](<https://devfeed.tech/tags/training.md>), [value](<https://devfeed.tech/tags/value.md>), [video](<https://devfeed.tech/tags/video.md>), [vision](<https://devfeed.tech/tags/vision.md>)

## AI overview

FOCUS reduces video understanding cost by deciding when to invoke a VLM and which visual regions to emphasize. A smaller VLM detects meaningful scene changes to trigger bursts of frames; patch-level differences combine with an attention prior to form a spatial focus map. During generation, that map boosts relevant vision tokens and attenuates others without removing tokens, while decoder cross-attention feeds back into the next burst's prior. In streaming evaluation on a 500-video EgoSchema subset using Qwen3-VL-8B, FOCUS achieved 74.8% accuracy with 25.4 calls per clip, compared with 74.6% and 55 calls for uniform sampling and 74.2% and 55 calls for BOLT. For egocentric captioning across four domains, it matched the best baseline's quality with 57% fewer invocations than uniform sampling.

## Source excerpt

arXiv:2611.01438v1 Announce Type: new Abstract: Processing every video frame through Vision-Language Models (VLMs) adds significant inference cost and limits their ability to comprehend real-time video. Existing approaches either sub-sample frames uniformly, ignoring content and missing important events, or prune visual tokens inside the VLM to cut per-call cost while still invoking the model on every sampled frame. We introduce FOCUS, a training-free three-layer framework that reduces the number of invocations and steers each one towards what matters. Temporal gating measures inter-frame dissimilarity in the patch-embedding space of a smaller VLM from the same family, triggering burst captures only when the scene changes in a semantically meaningful way. Patch-level dissimilarity is then accumulated across the burst and fused with a top-down attention prior into a spatial focus map that encodes where motion and semantic relevance coincide. That map calibrates the VLM's value cache during generation, progressively amplifying vision tokens in task-relevant regions and attenuating off-target ones, without removing any token from the context. The loop closes after each response: text-to-vision cross-attention is extracted from the decoder and fed back as the prior for the next burst, letting the system converge on semantically important regions over successive observations. We evaluate FOCUS on the 500-video EgoSchema subset in streaming mode, with a single Qwen3-VL-8B backbone shared by every method. It reaches 74.8% accuracy using 25.4 VLM calls per clip, against 74.6% with 55 calls for uniform sampling and 74.2% with 55 calls for BOLT. On egocentric video captioning across four domains, it matches the best baseline's quality while issuing 57% fewer VLM invocations than uniform sampling.