# Goal Retrieval versus Action-Value Ordering: Contrastive Critics under Best-of-$K$ Selection

DevFeed: [Goal Retrieval versus Action-Value Ordering: Contrastive Critics under Best-of-$K$ Selection](<https://devfeed.tech/articles/goal-retrieval-versus-action-value-ordering-contrastive-critics-under-best-of-k-selection-94742.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2607.27422>)

Author: Ayushman Singh, Siddharth Aphale

Published: 2026-10-22T04:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [ordering](<https://devfeed.tech/topics/ordering.md>), [information retrieval](<https://devfeed.tech/topics/information-retrieval.md>), [Robot Navigation](<https://devfeed.tech/topics/robot-navigation.md>)

Tags: [actions](<https://devfeed.tech/tags/actions.md>), [audit](<https://devfeed.tech/tags/audit.md>), [best-of](<https://devfeed.tech/tags/best-of.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [fail](<https://devfeed.tech/tags/fail.md>), [fixed](<https://devfeed.tech/tags/fixed.md>), [gap](<https://devfeed.tech/tags/gap.md>), [generators](<https://devfeed.tech/tags/generators.md>), [gives](<https://devfeed.tech/tags/gives.md>), [goals](<https://devfeed.tech/tags/goals.md>), [improvement](<https://devfeed.tech/tags/improvement.md>), [model](<https://devfeed.tech/tags/model.md>), [navigation](<https://devfeed.tech/tags/navigation.md>), [oracle](<https://devfeed.tech/tags/oracle.md>), [ordering](<https://devfeed.tech/tags/ordering.md>), [performance](<https://devfeed.tech/tags/performance.md>), [policy](<https://devfeed.tech/tags/policy.md>), [retrieval](<https://devfeed.tech/tags/retrieval.md>), [selection](<https://devfeed.tech/tags/selection.md>), [value](<https://devfeed.tech/tags/value.md>)

## AI overview

High goal-retrieval AUC does not guarantee that a critic will rank candidate actions well for best-of-K selection. Across eight OGBench tasks, raw and cosine contrastive critics achieved retrieval AUCs of 0.96-1.00, yet their progress ordering on navigation tasks was weaker and sometimes inverted. In an exact-value controlled problem, all three contrastive selectors had greater mean regret than random selection, with regret sometimes rising as K increased; TD-Q stayed close to the candidate-pool oracle. On PointMaze, the only task with a reliable learned value reference, TD-Q ordered actions substantially better on fixed queries, although executed-return differences were smaller and task-dependent. Matched controls suggest graded value supervision explained most of the improvement, while score bounds and model capacity had less effect. The authors recommend separately evaluating action ordering before deployment.

## Source excerpt

arXiv:2607.27422v2 Announce Type: replace Abstract: Goal-conditioned action generators can produce many plausible actions. Best-of-\(K\) selection then uses a critic to rank those actions for the current state and goal. Standard contrastive critics are trained and evaluated on a different ranking: distinguishing future goals from unrelated negatives for a fixed state--action pair. The unnormalized contrastive score is monotone in behavior-policy value at the population optimum. Cosine-normalized scores lack this guarantee, and finite trained networks may fail to recover the correct ordering. We audit the resulting gap using shared state--goal queries and frozen candidate pools, so critics differ only in their action ordering. Raw and cosine critics reach retrieval AUCs of \(0.96\)--\(1.00\) across eight OGBench tasks. Their progress ordering on navigation tasks is much weaker and sometimes inverted. In the exact-value controlled problem, all three contrastive selectors have higher mean regret than random selection. Their regret can increase with \(K\). TD-Q stays close to the pool oracle. PointMaze is the only task with a reliable learned value reference. TD-Q gives substantially better fixed-query ordering there, though the differences in executed return are smaller and vary by task. Matched controls suggest that graded value supervision accounts for most of the improvement. Changing score bounds or model capacity alone has much less effect. High retrieval performance does not establish whether a critic ranks actions correctly for best-of-\(K\) selection. That requires a separate action-ordering evaluation before deployment.