# Deciding Without Pointing: A Case Study of Whether a Decision Model Localizes the Anomalies It Detects

DevFeed: [Deciding Without Pointing: A Case Study of Whether a Decision Model Localizes the Anomalies It Detects](<https://devfeed.tech/articles/deciding-without-pointing-a-case-study-of-whether-a-decision-model-localizes-the-anomalies-it-detects-141716.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.21995>)

Author: Hsiang-Hung Lee

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [vlm](<https://devfeed.tech/topics/vlm.md>), [LLM evaluation / benchmarking](<https://devfeed.tech/topics/llm-evaluation-benchmarking.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>)

Tags: [attention](<https://devfeed.tech/tags/attention.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [case-study](<https://devfeed.tech/tags/case-study.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [decision-model](<https://devfeed.tech/tags/decision-model.md>), [defect](<https://devfeed.tech/tags/defect.md>), [detection](<https://devfeed.tech/tags/detection.md>), [few-shot](<https://devfeed.tech/tags/few-shot.md>), [image](<https://devfeed.tech/tags/image.md>), [industrial](<https://devfeed.tech/tags/industrial.md>), [inspection](<https://devfeed.tech/tags/inspection.md>), [llm](<https://devfeed.tech/tags/llm.md>), [location](<https://devfeed.tech/tags/location.md>), [map](<https://devfeed.tech/tags/map.md>), [model](<https://devfeed.tech/tags/model.md>), [multimodal](<https://devfeed.tech/tags/multimodal.md>), [question](<https://devfeed.tech/tags/question.md>), [qwen3-5](<https://devfeed.tech/tags/qwen3-5.md>), [qwen3-8](<https://devfeed.tech/tags/qwen3-8.md>), [qwen3-8-27b](<https://devfeed.tech/tags/qwen3-8-27b.md>), [region](<https://devfeed.tech/tags/region.md>)

## AI overview

Clef-flash detects industrial anomalies unevenly across the two benchmarks: image AUROC is 0.924 on MVTec AD and 0.668 on MVTec AD 2. On MVTec AD, its decision head's routing attention localizes poorly (mean per-image pixel AUROC 0.391), and blocking image tokens from that head does not change detection. Backbone attention can point to defects, but its maps depend on question wording and textual context; the larger Clef model also provides no useful localization from its decision head. The authors report recovering location through region-specific questions, head retraining, or reading backbone states, but none matches dedicated few-shot localizers.

## Source excerpt

arXiv:2611.21995v1 Announce Type: cross Abstract: Decision models can answer a typed question about an image with a probability per option, without generating text. In our case study of Clef-flash (post-trained Qwen3.5-9B backbone) on industrial inspection benchmarks, we ask whether it localizes the anomalies it detects. We obtained uneven results with zero-shot detection: per-category image AUROC 0.924 on MVTec AD and 0.668 on MVTec AD 2. On the former, the decision head's routing attention produces 0.391 mean per-image pixel AUROC, and hiding the image tokens from it leaves detection unchanged. The multimodal LLM's backbone attention does point at the defect, however it falls below a center prior when we change question. It falls below chance if we remove textual context in the question, partly due to backbone attention leaving the object, while the head's map hardly changes. The backbone's states carry the location when probed with normal images, on the test-split images, however, the decision depends on the defect region. A larger model, Clef, that comes with a post-trained Qwen3.8-27B backbone, gives no useful map in the decision head either. We found that it's possible to recover anomaly location by asking question specifically on an image region, retraining the head or reading the backbone, but none matches dedicated few-shot localizers.