# Where Do Vision-Language Models Get Left and Right? Position-Code Counterfactuals in Qwen2.5-VL, Qwen3-VL and InternVL3

DevFeed: [Where Do Vision-Language Models Get Left and Right? Position-Code Counterfactuals in Qwen2.5-VL, Qwen3-VL and InternVL3](<https://devfeed.tech/articles/where-do-vision-language-models-get-left-and-right-position-code-counterfactuals-in-qwen2-5-vl-qwen3-vl-and-internvl3-128794.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.14060>)

Author: Enhao Cui, Junzhi Yang, Yaoshun Lin, Yuhan Feng, Zijian Chen

Published: 2026-11-11T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [vlm](<https://devfeed.tech/topics/vlm.md>), [qwen](<https://devfeed.tech/topics/qwen.md>)

Tags: [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [position](<https://devfeed.tech/tags/position.md>), [right](<https://devfeed.tech/tags/right.md>), [vision-language-model](<https://devfeed.tech/tags/vision-language-model.md>), [vision-language-models](<https://devfeed.tech/tags/vision-language-models.md>)

## AI overview

Across Qwen2.5-VL, Qwen3-VL and InternVL3, mirroring the vision transformer's (ViT) horizontal position code often reproduces the answer to a horizontally flipped image on CV-Bench left-right questions (84.3-93.2% in the reported models). This supports the ViT code's role in left-right direction; mirroring language-model token positions or order changes few answers, though one Qwen2.5-VL test is described as weak. The result is not conclusive evidence of robust spatial reasoning: with two answer options, disruption can count as a flip, and a half-grid shift also flips many answers. On What'sUp, position-code mirroring behaves differently for on/under relations, and permutation analyses suggest errors involving object order and relation binding. The authors identify localization and use of a reference object as possible targets for repair.

## Source excerpt

arXiv:2611.14060v1 Announce Type: cross Abstract: A vision-language model may take left-right direction from the orientation of image content within patches, its vision transformer's (ViT) position code, or token positions and order in its language model (LLM). Ablations show the ViT code is necessary; we test sufficiency by mirroring only a position code. In Qwen2.5-VL-7B, Qwen3-VL-4B and InternVL3-2B, mirroring ViT width positions reproduces the flipped-image answer on 84.3-93.2% of CV-Bench left-right questions (Qwen2.5-VL-3B: 87.5%, partly a left fallback). With two options, this also counts disruption; a half-grid shift flips most answers too. With the ViT code intact, LLM-side id and token-order mirrors change few left-right answers in all three series (a weak id test in Qwen2.5-VL). In Qwen2.5-VL-7B the effect sits in the global-attention ViT blocks, mostly the last. On What'sUp, flipping content and token order, not the position code, keeps more on/under than left-right answers at the flipped-image answer. Post hoc analyses qualify these results. On CV-Bench items with located objects, random permutations flip 18.9-31.0 points more answers when reversing the objects' order than when keeping it. On What'sUp, in Qwen2.5-VL and Qwen3-VL, the share of the left-right changes made by random permutations that land on the mirror relation is mostly below chance. Counted as returns to the original answer, the What'sUp contrast of support with left-right relations is clear only in InternVL3-2B. Most remaining errors give the same relation whichever object is asked about, suggesting that repair may target relation binding, object localization or use of the reference object.