# On the Faithfulness of Visual Thinking: Measurement and Enhancement

DevFeed: [On the Faithfulness of Visual Thinking: Measurement and Enhancement](<https://devfeed.tech/articles/on-the-faithfulness-of-visual-thinking-measurement-and-enhancement-111868.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2510.23482>)

Author: Zujing Liu, Junwen Pan, Qi She, Yuan Gao, Guisong Xia

Published: 2026-11-02T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [vlm](<https://devfeed.tech/topics/vlm.md>), [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [Chain-of-thought](<https://devfeed.tech/topics/chain-of-thought.md>), [LLM evaluation / benchmarking](<https://devfeed.tech/topics/llm-evaluation-benchmarking.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>)

Tags: [automated](<https://devfeed.tech/tags/automated.md>), [chain-of-thought](<https://devfeed.tech/tags/chain-of-thought.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [enhancement](<https://devfeed.tech/tags/enhancement.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [learning](<https://devfeed.tech/tags/learning.md>), [measurement](<https://devfeed.tech/tags/measurement.md>), [metric](<https://devfeed.tech/tags/metric.md>), [rl](<https://devfeed.tech/tags/rl.md>), [sufficient](<https://devfeed.tech/tags/sufficient.md>), [thinking](<https://devfeed.tech/tags/thinking.md>), [vision](<https://devfeed.tech/tags/vision.md>), [vision-language-models](<https://devfeed.tech/tags/vision-language-models.md>)

## AI overview

The paper finds that visual content in reinforcement fine-tuned multimodal chain-of-thought traces can be inaccurate while answers remain correct: predictions change little when visual thoughts are intervened on, but change significantly when textual thoughts are. The authors attribute this to reward designs that encourage interleaved vision-text formatting without checking visual correctness. They introduce an LVLM-based metric for visual cue relevance and sufficiency, reporting that current traces often fail on both. Their annotation-free, plug-and-play Sufficient-Component Cause Model (SCCM) learning encourages minimal visual components sufficient to support the answer. Experiments across fine-grained perception and reasoning benchmarks report improved visual faithfulness; the abstract does not provide quantitative results.

## Source excerpt

arXiv:2510.23482v2 Announce Type: replace-cross Abstract: Recent large vision-language models (LVLMs) can generate vision-text multimodal chain-of-thought (MCoT) traces after reinforcement fine-tuning (RFT). However, we observe that the visual information in MCoT is often inaccurate even when answers are correct, and task accuracy remains nearly unchanged despite the visual information being corrupted. This indicates a lack of faithfulness in the vision part of MCoT reasoning. We attribute this to the RL reward design in RFT, which solely incentivizes the format of interleaved vision-text cues, encouraging the model to incorporate visual information into its text reasoning steps without considering its correctness. In this paper, we first probe MCoT faithfulness by measuring how much the prediction changes when its visual and textual thoughts are intervened. Surprisingly, the model's predictions remain nearly unchanged under visual intervention but change significantly under textual intervention, indicating that the visual evidence is largely ignored. To further analyze the visual information, we introduce an automated LVLM-based evaluation metric that quantifies the faithfulness of visual cues from two perspectives: relevance and sufficiency. Our evaluation reveals that the visual information in current MCoT traces is simultaneously irrelevant and insufficient. To address this issue, we propose Sufficient-Component Cause Model (SCCM) learning. This approach encourages the MCoT to generate sufficient yet minimal visual components that are independently capable of leading to the correct answer. The proposed SCCM is annotation-free and plug-and-play, compatible with various RFT for MCoT. Empirical results demonstrate that SCCM substantially improves the visual faithfulness across a suite of fine-grained perception and reasoning benchmarks. Code is available at https://github.com/EugeneLiu01/Faithful_Thinking_with_Image.