# Zero-Shot VLA Adaptation with Retrieved Guidance

DevFeed: [Zero-Shot VLA Adaptation with Retrieved Guidance](<https://devfeed.tech/articles/zero-shot-vla-adaptation-with-retrieved-guidance-125277.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.10894>)

Author: Subin Kim, Anh-Quan Pham, Dinesh Jayaraman

Published: 2026-11-10T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Humanoid Robots](<https://devfeed.tech/topics/humanoid-robots.md>), [Enhancing AI model performance](<https://devfeed.tech/topics/enhancing-ai-model-performance.md>), [Retrieval Augmented Generation (RAG)](<https://devfeed.tech/topics/retrieval-augmented-generation-rag.md>)

Tags: [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-ro](<https://devfeed.tech/tags/cs-ro.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [domain](<https://devfeed.tech/tags/domain.md>), [flow-matching](<https://devfeed.tech/tags/flow-matching.md>), [guidance](<https://devfeed.tech/tags/guidance.md>), [humanoid](<https://devfeed.tech/tags/humanoid.md>), [inference](<https://devfeed.tech/tags/inference.md>), [rag](<https://devfeed.tech/tags/rag.md>), [real-world](<https://devfeed.tech/tags/real-world.md>), [retrieval](<https://devfeed.tech/tags/retrieval.md>), [retrieval-augmented-generation](<https://devfeed.tech/tags/retrieval-augmented-generation.md>), [robotics](<https://devfeed.tech/tags/robotics.md>), [sim](<https://devfeed.tech/tags/sim.md>), [sim-to-real-transfer](<https://devfeed.tech/tags/sim-to-real-transfer.md>), [simulation](<https://devfeed.tech/tags/simulation.md>), [source](<https://devfeed.tech/tags/source.md>), [strategy](<https://devfeed.tech/tags/strategy.md>), [target](<https://devfeed.tech/tags/target.md>), [training-data](<https://devfeed.tech/tags/training-data.md>), [vla](<https://devfeed.tech/tags/vla.md>), [zero-shot](<https://devfeed.tech/tags/zero-shot.md>)

## AI overview

RAGu uses retrieved demonstrations to help pretrained Vision-Language-Action models handle domain shifts such as changes in camera viewpoint, background, object identity, or robot morphology, without retraining the base model. It applies geometric calibration to adapt source demonstrations to a target setup, then uses coarse-to-fine retrieval to select a trajectory compatible with the VLA and guide its flow-matching inference. Across real-world, simulation, sim-to-real, and cross-embodiment humanoid manipulation evaluations, the authors report real-world success improving from 20% to 80% and better performance than the closest related baselines in simulation.

## Source excerpt

arXiv:2611.10894v1 Announce Type: cross Abstract: Pre-training large parametric "Vision-Language-Action" (VLA) models on large demonstration datasets has recently yielded promising results for general-purpose manipulation. However, these models still struggle under even small domain shifts relative to the training data, such as changes in camera viewpoint, background, object identity, or robot morphology. Motivated by the success of Retrieval-Augmented Generation (RAG) in language models, we pursue retrieval to boost VLA performance. Specifically, we seek to leverage retrieval from domain-mismatched, "in-the-wild" demonstration datasets like DROID to guide a VLA's behavior in a new target domain. To achieve this, we introduce RAGu, which overcomes robotics-specific challenges for RAG: (1) It incorporates simple geometric calibration to transform source demonstrations into the target setup and reduce cross-domain differences. (2) It uses a coarse-to-fine retrieval strategy that selects, from a large pool of pre-warped candidates, the trajectory most compatible with the base VLA. (3) It guides the VLA's flow-matching inference procedure rather than seeking to retrain the base model or expand its limited context window. We evaluate RAGu across real-world, simulation, sim-to-real transfer, and cross-embodiment humanoid manipulation. We find it improves real-world VLA success rates from 20% to 80% and outperforms the closest-related baselines in sim. Project website: raguvla.github.io