# Racing the Clock: Teaching VLNs Budget-Conditioned Object Search

DevFeed: [Racing the Clock: Teaching VLNs Budget-Conditioned Object Search](<https://devfeed.tech/articles/racing-the-clock-teaching-vlns-budget-conditioned-object-search-135066.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.17442>)

Author: Maitree Meher, Siming He, S. Shankar Sastry, Jitendra Malik, Claire Tomlin, Jonas Frey

Published: 2026-11-16T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Robot Navigation](<https://devfeed.tech/topics/robot-navigation.md>), [AI Agent](<https://devfeed.tech/topics/ai-agent.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [agent](<https://devfeed.tech/tags/agent.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [behavior](<https://devfeed.tech/tags/behavior.md>), [budget](<https://devfeed.tech/tags/budget.md>), [building](<https://devfeed.tech/tags/building.md>), [clock](<https://devfeed.tech/tags/clock.md>), [critical](<https://devfeed.tech/tags/critical.md>), [cross](<https://devfeed.tech/tags/cross.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-ro](<https://devfeed.tech/tags/cs-ro.md>), [effective](<https://devfeed.tech/tags/effective.md>), [environment](<https://devfeed.tech/tags/environment.md>), [exploration](<https://devfeed.tech/tags/exploration.md>), [explore](<https://devfeed.tech/tags/explore.md>), [fixed](<https://devfeed.tech/tags/fixed.md>), [full](<https://devfeed.tech/tags/full.md>), [how-to](<https://devfeed.tech/tags/how-to.md>), [instead](<https://devfeed.tech/tags/instead.md>), [object](<https://devfeed.tech/tags/object.md>), [search](<https://devfeed.tech/tags/search.md>), [teaching](<https://devfeed.tech/tags/teaching.md>)

## AI overview

The paper finds that supervision for budget-conditioned object search is more effective when it teaches exploration under partial observability than when it imitates routes optimized with full map and target knowledge. It compares a privileged TSP expert with a room-level Bellman expert using cross-scene semantic priors. Although the TSP expert performs substantially better itself, on held-out HM3D scenes the Bellman-supervised VLN consistently outperforms the TSP-supervised model, with a larger advantage at longer budgets. At 2,000 steps (500 m), they find 71.4% and 65.5% of reachable target instances, respectively, versus 44.8% for a pretrained VLN.

## Source excerpt

arXiv:2611.17442v1 Announce Type: cross Abstract: A rescue robot entering a building with thirty seconds on the clock should sprint toward the single most promising room; given ten minutes, it should instead explore a broader tour. Yet vision-language-navigation (VLN) models are typically evaluated under a fixed episode limit that the policy never observes, even though time-critical missions require behavior to adapt to the available budget. We study budget-conditioned object search: given a target category and an explicit action budget, an agent must find as many distinct target instances as possible before the budget expires. We investigate how to supervise VLNs for this task by comparing a privileged TSP expert, which plans near-optimal routes using the full map and true target locations, with a room-level Bellman expert that maximizes expected return under partial observability using high-level cross-scene semantic priors. Despite the TSP expert achieving substantially higher expert performance, its demonstrations are less effective for learning exploration: on held-out HM3D scenes, the Bellman-supervised VLN consistently outperforms the TSP-supervised VLN, with the advantage becoming more pronounced at longer budgets. At 2000 steps (500m), the Bellman-supervised VLN finds 71.4% of reachable target instances, compared with 65.5% for the TSP-supervised VLN and 44.8% for the pretrained VLN. These results highlight a distinction between optimal routing and optimal exploration: effective exploration supervision should optimize decisions under partial observability rather than imitate routes that are optimal only once the environment is fully known.