# Computer vision

Computer vision computes properties of the three-dimensional world from digital images.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## SteerTrace: Tracing How Target Attacks Steer World Action Models

DevFeed: [SteerTrace: Tracing How Target Attacks Steer World Action Models](<https://devfeed.tech/articles/steertrace-tracing-how-target-attacks-steer-world-action-models-145575.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.25719>)

Author: Yukun Dai, Tianshi Wang, Fengling Li, Jingjing Li, Xin Li, Lei Zhu

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [World models](<https://devfeed.tech/topics/world-models.md>), [Computer vision](<https://devfeed.tech/topics/computer-vision.md>), [Neural Network](<https://devfeed.tech/topics/neural-network.md>), [Physical AI](<https://devfeed.tech/topics/physical-ai.md>)

Tags: [a-safety](<https://devfeed.tech/tags/a-safety.md>), [actions](<https://devfeed.tech/tags/actions.md>), [attack](<https://devfeed.tech/tags/attack.md>), [attacks](<https://devfeed.tech/tags/attacks.md>), [bounded](<https://devfeed.tech/tags/bounded.md>), [budget](<https://devfeed.tech/tags/budget.md>), [camera](<https://devfeed.tech/tags/camera.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [effect](<https://devfeed.tech/tags/effect.md>), [experts](<https://devfeed.tech/tags/experts.md>), [ffn](<https://devfeed.tech/tags/ffn.md>), [frame](<https://devfeed.tech/tags/frame.md>), [future](<https://devfeed.tech/tags/future.md>), [inside](<https://devfeed.tech/tags/inside.md>), [layers](<https://devfeed.tech/tags/layers.md>), [model](<https://devfeed.tech/tags/model.md>), [models](<https://devfeed.tech/tags/models.md>), [noise](<https://devfeed.tech/tags/noise.md>), [target](<https://devfeed.tech/tags/target.md>), [tracing](<https://devfeed.tech/tags/tracing.md>), [world](<https://devfeed.tech/tags/world.md>)

### AI overview

SteerTrace reports that a single per-model perturbation bounded at 4/255, shared across frames, tasks, and camera views, steers actions toward an attacker-chosen target across eight world action models on LIBERO and RoboTwin, bringing task success close to zero. Same-budget noise and BadWAM cause little shift toward the target. Tracing the effect shows different dependencies: under the standard attack, restoring future tokens removes 61-94% of the targeted effect in five models whose action experts attend to them. When predicted futures are kept close to clean predictions, current-frame tokens become more important in three models: restoring them removes 87-99% of the effect, versus 9-15% for future tokens. In six models, restoring 0.36% of video-expert FFN neurons in layers 10-29 removes 72-96% of the effect. The abstract also reports that residual directions 2-16 contain only 11-18% of shift energy, yet removing them cancels most of the effect.

### Source excerpt

arXiv:2611.25719v1 Announce Type: cross Abstract: World action models (WAMs) predict future videos and actions with one backbone, and their imagined future is often seen as a safety signal. We introduce SteerTrace, a targeted universal attack on WAMs that also traces its effect inside the model. One $\ell_\infty$-bounded (4/255) perturbation per model, shared across frames, tasks, and camera views, pushes the executed action toward an attacker-chosen target. On eight WAMs from four families (FastWAM, Faster-WAM, OpenWAM and LingBot-VA) on LIBERO and RoboTwin, it drives task success close to zero and moves the attacked action toward the target. Same-budget noise and BadWAM produce little action shift in the target direction. We use the signed targeted effect to measure how much of the effect each intervention removes or reproduces. Under the standard attack, restoring future tokens removes 61-94% of the targeted effect in the five models whose action experts attend to them. When the attack is constrained to keep the predicted future close to the clean prediction, the targeted effect can become more dependent on current-frame tokens. In three of these five models, restoring current-frame tokens removes 87-99% of the effect, compared with 9-15% for future tokens. In six models, restoring just 0.36% of the FFN neurons in layers 10-29 of the video expert removes 72-96% of the targeted effect. Residual directions 2-16 hold only 11-18% of the shift energy, yet removing them cancels most of the targeted effect.

## Physical Laws without Physics Loss? Probing Learned Dynamics in Video World Models through Forced Response

DevFeed: [Physical Laws without Physics Loss? Probing Learned Dynamics in Video World Models through Forced Response](<https://devfeed.tech/articles/physical-laws-without-physics-loss-probing-learned-dynamics-in-video-world-models-through-forced-response-145081.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.25626>)

Author: Yutao Li, Chen Wang, Wangmeng Zuo, Hui Li

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [World models](<https://devfeed.tech/topics/world-models.md>), [Computer vision](<https://devfeed.tech/topics/computer-vision.md>), [Generative AI](<https://devfeed.tech/topics/generative-ai.md>)

Tags: [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [flow-matching](<https://devfeed.tech/tags/flow-matching.md>), [image-to-video](<https://devfeed.tech/tags/image-to-video.md>), [laws](<https://devfeed.tech/tags/laws.md>), [motion](<https://devfeed.tech/tags/motion.md>), [object](<https://devfeed.tech/tags/object.md>), [physics](<https://devfeed.tech/tags/physics.md>), [probing](<https://devfeed.tech/tags/probing.md>), [response](<https://devfeed.tech/tags/response.md>), [training](<https://devfeed.tech/tags/training.md>), [video](<https://devfeed.tech/tags/video.md>), [world-models](<https://devfeed.tech/tags/world-models.md>)

### AI overview

The study tests whether video world models learn dynamics beyond replay by shaking the support of suspended objects and evaluating forced responses. Six image-to-video backbones are adapted with visual and numeric conditioning using only flow-matching loss. All six reproduce linear response, including resonance, and five capture higher-order effects such as nonlinear softening, despite softening accounting for only 0.74% of the swing's squared displacement in training. Wan generalizes to unseen excitations, objects, poses, and scenes; counterfactual tests suggest it does more than replay training videos. Extensions to a double pendulum and rocking block find sustained dynamics are learned more faithfully than brief impacts. The excerpt also cautions that learned dynamics may not emerge when state estimation or propagation fails.

### Source excerpt

arXiv:2611.25626v1 Announce Type: cross Abstract: Do video world models learn physical laws without a physics loss, or merely imitate motion? Overall motion and appearance scores cannot settle it. We investigate this through nonlinear forced response: an excitation shakes the support of a suspended object. We introduce dual visual-numeric conditioning and adapt six image-to-video backbones to predict this motion using only the flow-matching loss. Wan predicts motion accurately and generalizes to unseen excitations, objects, poses, and scenes. Counterfactual tests suggest capabilities beyond replaying training videos. But what laws have they learned? We design three tiers of tests using Taylor expansions of the dynamics and geometry. All six reproduce the linear response, including resonance; five capture higher-order dynamics, including nonlinear softening. Remarkably, softening is barely present in training: only 0.74% of the swing's squared displacement. Extensions to a double pendulum and a rocking block show sustained dynamics are learned more faithfully than brief impacts. Yet learned dynamics may fail to appear when state estimation or propagation goes wrong. We turn these tests and findings into strategies to improve video training and generation.

## Fixing the Twin, Not the Detector: Sim-to-Real Roadside LiDAR Detection from Digital-Twin Data

DevFeed: [Fixing the Twin, Not the Detector: Sim-to-Real Roadside LiDAR Detection from Digital-Twin Data](<https://devfeed.tech/articles/fixing-the-twin-not-the-detector-sim-to-real-roadside-lidar-detection-from-digital-twin-data-145130.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.26035>)

Author: KuanLin Chen, Feng-jui Kuo

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Computer vision](<https://devfeed.tech/topics/computer-vision.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>)

Tags: [2026](<https://devfeed.tech/tags/2026.md>), [annotation](<https://devfeed.tech/tags/annotation.md>), [challenge](<https://devfeed.tech/tags/challenge.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [detection](<https://devfeed.tech/tags/detection.md>), [digital-twin](<https://devfeed.tech/tags/digital-twin.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [lidar](<https://devfeed.tech/tags/lidar.md>), [model](<https://devfeed.tech/tags/model.md>), [pipeline](<https://devfeed.tech/tags/pipeline.md>), [point](<https://devfeed.tech/tags/point.md>), [sensor](<https://devfeed.tech/tags/sensor.md>), [sim](<https://devfeed.tech/tags/sim.md>), [sim-to-real](<https://devfeed.tech/tags/sim-to-real.md>), [site](<https://devfeed.tech/tags/site.md>), [size](<https://devfeed.tech/tags/size.md>), [static](<https://devfeed.tech/tags/static.md>), [statistics](<https://devfeed.tech/tags/statistics.md>), [synthetic](<https://devfeed.tech/tags/synthetic.md>), [synthetic-data](<https://devfeed.tech/tags/synthetic-data.md>), [test](<https://devfeed.tech/tags/test.md>)

### AI overview

A synthetic-only CenterPoint pipeline achieved first place in the UrbanTwin challenge's LUMPI track (0.5541 combined score) and fifth in V2X-Real (0.8102), with most gains attributed to correcting twin-to-site differences. The approach aligns coordinates, point density, and box conventions; adds pedestrians, riders, and unlabeled street clutter; and rebuilds twin objects at real-world size and density. Non-learned resampling raised LUMPI realism from 0.589 to 0.947, close to the 0.949 similarity between real frames. Label-free post-processing uses recurring static structures and real-object point statistics. Systematic corrections generally retained one-quarter to three-quarters of validation gains on test data, while choices fitted to the few validation objects did not transfer. Detectors were trained only on synthetic data using a single 8 GB laptop GPU.

### Source excerpt

arXiv:2611.26035v1 Announce Type: cross Abstract: Roadside LiDAR detectors are expensive to train on real labels, and digital twins promise to replace these labels, but a detector trained on a twin meets a different sensor, different objects and different annotation conventions on the real site. We describe the pipeline that placed first in the LUMPI track of the UrbanTwin Sim2Real LiDAR Challenge (IEEE BigData 2026 Cup, 0.5541 combined score) and fifth on its V2X-Real track (0.8102). Every detector is a CenterPoint model trained only on synthetic data, on a single 8 GB laptop GPU. Most of the gain came from correcting differences between the twin and the real site rather than from the detector. We align the twin's coordinates, point density and box convention with the test sensor, generate the pedestrians and riders that the twins lack, plus street clutter as unlabeled negatives, and rebuild twin objects at real size and real point density. A non-learned resampling to the per-cell point density of unlabeled real frames lifts LUMPI realism from 0.589 to 0.947, against 0.949 between real frames. Label-free post-processing then exploits what a fixed roadside sensor guarantees: static structures recur in every frame, and real objects have characteristic point statistics. Reporting the validation and test gain of each component side by side, we find that corrections of systematic sim-to-real differences transfer to the test set, typically keeping a quarter to three quarters of their validation gain, while choices that fit the few objects of a validation set do not.

## Digitization of Analogue Gauges using Computer Vision

DevFeed: [Digitization of Analogue Gauges using Computer Vision](<https://devfeed.tech/articles/digitization-of-analogue-gauges-using-computer-vision-145647.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.26262>)

Author: Erico Kayumba Mutonkole, Olasupo Ajayi, Omowunmi Isafiade

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Computer vision](<https://devfeed.tech/topics/computer-vision.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [OpenCV](<https://devfeed.tech/topics/opencv.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [image recognition](<https://devfeed.tech/topics/image-recognition.md>), [Image processing](<https://devfeed.tech/topics/image-processing.md>)

Tags: [automation](<https://devfeed.tech/tags/automation.md>), [computer-vision](<https://devfeed.tech/tags/computer-vision.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [dataset](<https://devfeed.tech/tags/dataset.md>), [deep-learning](<https://devfeed.tech/tags/deep-learning.md>), [gauge](<https://devfeed.tech/tags/gauge.md>), [manufacturing](<https://devfeed.tech/tags/manufacturing.md>), [opencv](<https://devfeed.tech/tags/opencv.md>), [residual](<https://devfeed.tech/tags/residual.md>), [web-scraping](<https://devfeed.tech/tags/web-scraping.md>)

### AI overview

The study reports that OpenCV with ResNet18 achieved 97.3% accuracy in reading analogue pressure gauges, compared with 64.4% for OpenCV with VGG19 and 63.8% for OpenCV with Scikit-Image. The comparison used 1,848 gauge images covering 0-10 BAR, assembled through web scraping, synthetic generation, and augmentation. These results favor the ResNet18 approach within the reported evaluation; the supplied abstract does not describe deployment performance.

### Source excerpt

arXiv:2611.26262v1 Announce Type: cross Abstract: Analogue gauges remain prevalent in manufacturing industries for measuring physical quantities, yet manual monitoring by operators introduces human error and operational inefficiency. This research developed and evaluated an automated gauge reading system using computer vision and deep learning techniques. Three hybrid approaches were implemented and compared: OpenCV with Scikit-Image (traditional computer vision), OpenCV with ResNet18, and OpenCV with VGG19 (deep learning approaches). A comprehensive dataset of 1,848 pressure gauge images (0-10 BAR range) was curated through web scraping, synthetic generation, and augmentation techniques. The experimental results demonstrate that ResNet18 significantly outperforms the alternatives, achieving an accuracy of 97.3%, compared to VGG19's 64.4% and Scikit-Image's 63.8%. The study validates the effectiveness of residual learning architectures for industrial gauge digitization, providing a robust foundation for Industry 5.0 automation applications.

## From Portraits to Pathology: Probing Diagnostic Overreach in Vision-Language Models

DevFeed: [From Portraits to Pathology: Probing Diagnostic Overreach in Vision-Language Models](<https://devfeed.tech/articles/from-portraits-to-pathology-probing-diagnostic-overreach-in-vision-language-models-145368.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2609.05540>)

Author: Karan Dua, Amit Agarwal, Hitesh Laxmichand Patel, Hansa Meghwani, Jyotika Singh, Ranjeet Gupta, Graham Horwood, Tao Sheng, Avi Sil, Sujith Ravi, Dan Roth

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [vlm](<https://devfeed.tech/topics/vlm.md>), [Computer vision](<https://devfeed.tech/topics/computer-vision.md>)

Tags: [adhd](<https://devfeed.tech/tags/adhd.md>), [anxiety](<https://devfeed.tech/tags/anxiety.md>), [appearance](<https://devfeed.tech/tags/appearance.md>), [computer-vision](<https://devfeed.tech/tags/computer-vision.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [dataset](<https://devfeed.tech/tags/dataset.md>), [diagnostic](<https://devfeed.tech/tags/diagnostic.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [guardrail](<https://devfeed.tech/tags/guardrail.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [parity](<https://devfeed.tech/tags/parity.md>), [safety](<https://devfeed.tech/tags/safety.md>), [semantic](<https://devfeed.tech/tags/semantic.md>), [vision-language-models](<https://devfeed.tech/tags/vision-language-models.md>), [vision-language-models-vlms](<https://devfeed.tech/tags/vision-language-models-vlms.md>)

### AI overview

The study finds that vision-language models can infer ASD, ADHD, major depressive disorder, and generalized anxiety disorder from facial expressions, despite appearance alone being insufficient for diagnosis. On PARITY, a demographically balanced dataset designed to vary expressions while keeping identity and other visual factors as constant as possible, 27 open- and closed-weight models showed inconsistent abstention across expressions, conditions, and task formats. The problem was widespread among open-weight models and also appeared in safety-trained frontier models that abstained on single portraits but made judgments when comparing portraits of the same person. Clinical instructions did not reliably prevent such judgments, and a guardrail model rarely flagged paired requests. Most baseline open-model refusals also failed to explain both why facial evidence is insufficient and why clinical assessment is needed. The authors warn that unsupported judgments and authoritative-sounding explanations may cause distress, reinforce stigma, or offer false reassurance.

### Source excerpt

arXiv:2609.05540v3 Announce Type: replace-cross Abstract: Many medical conditions require clinical assessment of symptoms, history, and functioning rather than judgments from appearance. We study whether vision-language models (VLMs) abstain from facial attribution of autism spectrum disorder (ASD), attention-deficit/hyperactivity disorder (ADHD), major depressive disorder (MDD), and generalized anxiety disorder (GAD). To support this study, we construct PARITY (Paired Assessment with Reused Identity), a demographically balanced dataset for testing whether changing facial expressions alters diagnostic judgments, while holding identity and other visual factors as constant as possible. Our comprehensive evaluation of 27 open- and closed-weight models reveals inconsistent abstention across expressions, conditions, and task formats. Models turn facial expressions into unsupported medical judgments: this behavior is widespread among open-weight VLMs and extends to safety-trained frontier models that abstain on individual portraits yet make diagnostic judgments when comparing portraits of the same identity. Safeguard checks reveal further gaps: clinical instructions do not consistently prevent these judgments, and a guardrail model rarely flags paired requests as unsafe. Semantic analysis further shows that most baseline open-model refusals fail to explain both the insufficiency of facial evidence and the need for clinical assessment. Unsupported judgments and seemingly authoritative explanations can cause distress, reinforce stigma, and offer false reassurance, underscoring the need for stronger safety measures in model training and deployment.

## Beyond Entity Recognition: Vision-Centric Knowledge Visual Question Answering

DevFeed: [Beyond Entity Recognition: Vision-Centric Knowledge Visual Question Answering](<https://devfeed.tech/articles/beyond-entity-recognition-vision-centric-knowledge-visual-question-answering-145570.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.25680>)

Author: Jiatong Ma, Yuchen Liu, Longteng Guo, Yichen Yuan, Yuze Wu, Dongze Hao, Ru Zhen, Yanhao Zhang, Haonan Lu, Jing Liu

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [vlm](<https://devfeed.tech/topics/vlm.md>), [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [image recognition](<https://devfeed.tech/topics/image-recognition.md>), [Computer vision](<https://devfeed.tech/topics/computer-vision.md>), [knowledge-engineering](<https://devfeed.tech/topics/knowledge-engineering.md>)

Tags: [acquisition](<https://devfeed.tech/tags/acquisition.md>), [agentic](<https://devfeed.tech/tags/agentic.md>), [agentic-retrieval](<https://devfeed.tech/tags/agentic-retrieval.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [bottleneck](<https://devfeed.tech/tags/bottleneck.md>), [complex](<https://devfeed.tech/tags/complex.md>), [cross](<https://devfeed.tech/tags/cross.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [dataset](<https://devfeed.tech/tags/dataset.md>), [external](<https://devfeed.tech/tags/external.md>), [knowledge](<https://devfeed.tech/tags/knowledge.md>), [knowledge-base](<https://devfeed.tech/tags/knowledge-base.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [large-language-models](<https://devfeed.tech/tags/large-language-models.md>), [multimodal](<https://devfeed.tech/tags/multimodal.md>), [open-web](<https://devfeed.tech/tags/open-web.md>), [question](<https://devfeed.tech/tags/question.md>), [reasoning](<https://devfeed.tech/tags/reasoning.md>), [retrieval](<https://devfeed.tech/tags/retrieval.md>), [scale](<https://devfeed.tech/tags/scale.md>), [vision](<https://devfeed.tech/tags/vision.md>)

### AI overview

VC-VQA benchmarks vision-centric knowledge reasoning in multimodal large language models, including how they acquire and use visual evidence across multiple reasoning stages. It distinguishes internal from external images and pre-retrieval from post-retrieval visual information; tasks include retrieving external visual evidence, spotting clues before retrieval, and localizing, verifying, or matching retrieved knowledge to an input image. Its controlled multimodal knowledge base provides traceable textual and visual evidence. The authors evaluate 21 models across four settings: no external knowledge, provided evidence, knowledge-base retrieval, and open-web search. They report that models still struggle, with fine-grained visual knowledge acquisition the main bottleneck, while agentic retrieval points to the value of multi-step planning and cross-modal integration.

### Source excerpt

arXiv:2611.25680v1 Announce Type: cross Abstract: We propose VC-VQA, a large-scale benchmark for vision-centric knowledge reasoning, aiming to systematically evaluate the capabilities of Multimodal Large Language Models (MLLMs) in fine-grained visual knowledge acquisition and complex multi-stage reasoning. Unlike existing benchmarks that primarily treat visual information as entity recognition and rely on text-based external knowledge, VC-VQA explicitly models the role of visual information by considering both its source (internal vs. external images) and stage of involvement (pre-retrieval vs. post-retrieval). Specifically, VC-VQA requires models to retrieve external visual evidence, identify visual clues before retrieval, and localize, verify, or match retrieved knowledge against the internal image. We further construct a controlled multimodal knowledge base with traceable textual and visual evidence for each sample. We evaluate 21 representative MLLMs under four settings: no external knowledge, with provided evidence, knowledge base retrieval, and open-web search. Experimental results lead to three key findings: current models still struggle with vision-centric knowledge reasoning; fine-grained visual knowledge acquisition remains the primary bottleneck; and agentic retrieval highlights the value of multi-step planning and cross-modal knowledge integration. Overall, VC-VQA provides a more challenging benchmark for evaluating vision-centric knowledge reasoning in MLLMs. Our code and dataset are available at https://github.com/CASIA-IVA-Lab/VC-VQA.

## AeroRSI: Recursive Self-Improvement for Aerial Vision-Language Navigation

DevFeed: [AeroRSI: Recursive Self-Improvement for Aerial Vision-Language Navigation](<https://devfeed.tech/articles/aerorsi-recursive-self-improvement-for-aerial-vision-language-navigation-142752.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.24077>)

Author: Shiwei Wu, Yuwen Pan, Zhi Li, Yaowen Li, Yan Zhou, Xuemiao Xu, Yu Liu

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Robot Navigation](<https://devfeed.tech/topics/robot-navigation.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Computer vision](<https://devfeed.tech/topics/computer-vision.md>)

Tags: [behavior](<https://devfeed.tech/tags/behavior.md>), [code](<https://devfeed.tech/tags/code.md>), [continuation](<https://devfeed.tech/tags/continuation.md>), [cost](<https://devfeed.tech/tags/cost.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-ro](<https://devfeed.tech/tags/cs-ro.md>), [distilling](<https://devfeed.tech/tags/distilling.md>), [failure](<https://devfeed.tech/tags/failure.md>), [inference](<https://devfeed.tech/tags/inference.md>), [natural-language](<https://devfeed.tech/tags/natural-language.md>), [navigation](<https://devfeed.tech/tags/navigation.md>), [policy](<https://devfeed.tech/tags/policy.md>), [raise](<https://devfeed.tech/tags/raise.md>), [recursive-self-improvement](<https://devfeed.tech/tags/recursive-self-improvement.md>), [reduce](<https://devfeed.tech/tags/reduce.md>), [self-improvement](<https://devfeed.tech/tags/self-improvement.md>), [supervision](<https://devfeed.tech/tags/supervision.md>), [uavs](<https://devfeed.tech/tags/uavs.md>), [vision](<https://devfeed.tech/tags/vision.md>), [vision-language-action](<https://devfeed.tech/tags/vision-language-action.md>), [vla](<https://devfeed.tech/tags/vla.md>)

### AI overview

AeroRSI improves aerial vision-language navigation by having a policy learn from its own flight failures. In each round, it anchors supervision to failures reached by the current policy, tests candidate repairs by continuing the flight from them, then distills verified repairs while preserving behavior elsewhere. This lets even sampled actions and rule-based proposals that fail when directly imitated provide useful supervision. Repeating the process also reveals failures previously hidden because episodes stop at the first failure. On TravelUAV, three rounds increased a supervised 2B-parameter policy's success rate from 46.4% to 67.5% and halved collisions; the paper reports a 10.6-point gain over prior state of the art from the same initialization, without extra human labels or inference cost.

### Source excerpt

arXiv:2611.24077v1 Announce Type: cross Abstract: Vision-language-action (VLA) models enable unmanned aerial vehicles (UAVs) to follow natural-language instructions, yet still fail frequently in closed-loop flight. Correcting these failures requires repairs, i.e., actions that avoid them at the states the policy itself reaches, which human pilots cannot label at scale and rule-based planners cannot supply reliably. In this paper, we present AeroRSI, a recursive self-improvement framework in which an aerial navigation policy iteratively learns from its own failures. Our key insight is that a failing policy should judge its own repairs by flying on from them. Judged in this way, its sampled actions and even rule-based proposals that hurt it when imitated directly become reliable supervision. Moreover, an episode terminates at its first failure, concealing subsequent failures until the earlier ones are repaired. AeroRSI thus renews its supervision in every round, anchoring the failures of the current policy, verifying repairs by its continuation, and distilling them while keeping its behavior elsewhere. On TravelUAV, three rounds raise the success rate of a supervised 2B-parameter policy from 46.4% to 67.5% and reduce collisions by half, surpassing the previous state of the art by 10.6 points from the same initialization without extra human labels or inference cost. Code and models are available at https://github.com/starsholic/AeroRSI.

## Structured Extraction of Diffraction Data from Covalent Organic Frameworks

DevFeed: [Structured Extraction of Diffraction Data from Covalent Organic Frameworks](<https://devfeed.tech/articles/structured-extraction-of-diffraction-data-from-covalent-organic-frameworks-142690.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.23680>)

Author: Shenghan Gao, Zhiling Zheng

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Computer vision](<https://devfeed.tech/topics/computer-vision.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>)

Tags: [computer-vision](<https://devfeed.tech/tags/computer-vision.md>), [cond-mat-mtrl-sci](<https://devfeed.tech/tags/cond-mat-mtrl-sci.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [data](<https://devfeed.tech/tags/data.md>), [large-language-model](<https://devfeed.tech/tags/large-language-model.md>), [patterns](<https://devfeed.tech/tags/patterns.md>), [pipeline](<https://devfeed.tech/tags/pipeline.md>)

### AI overview

A vision LLM identifies diffraction plots and curves in varied PXRD figures and sketches their traces; classical computer vision then refines each trace to pixel-level accuracy. Applied to 3,954 COF articles, the pipeline yielded 11,445 calibrated curves for the open COF-PEAK database. Against hand digitization of 60 panels, peak positions differed by a median of 0.07 degrees. The curves retain information beyond the few peak angles typically reported in papers and could support analysis of how peak prominence, a possible marker of crystallinity, varies with synthesis conditions.

### Source excerpt

arXiv:2611.23680v1 Announce Type: cross Abstract: Powder X-ray diffraction (PXRD) is the primary structural evidence for covalent organic frameworks (COFs) and many other materials. Yet decades of measured patterns exist only as images in published figures. Automatic recovery is difficult, because PXRD figures vary widely in format, from a single curve to crowded stacks with labels, insets, and neighboring plots that challenge traditional computer vision. This work presents a pipeline in which a vision large language model (LLM) interprets the layout of each figure, locating the diffraction plot and its curves and sketching a coarse trace of each, while classical computer vision (CV) refines every trace to pixel-level accuracy. Applied to 3,954 COF articles, the pipeline produces 11,445 calibrated diffraction curves, released in the open database COF-PEAK. Compared with a hand digitization of 60 panels, extracted peak positions differ by a median of only 0.07 degrees. Whereas papers usually report only a few peak angles, which limits further analysis, the digitized curves enable tracking how peak prominence, a possible marker of crystallinity, changes with synthesis conditions. Vision LLMs and classical CV can thus turn such figures into reusable experimental data.

## RoboTeam: Benchmarking Vision-Based Multi-Agent Control of Many Robot Arms in Real Time

DevFeed: [RoboTeam: Benchmarking Vision-Based Multi-Agent Control of Many Robot Arms in Real Time](<https://devfeed.tech/articles/roboteam-benchmarking-vision-based-multi-agent-control-of-many-robot-arms-in-real-time-139969.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.22138>)

Author: Changxuan Fan, Zhongwei Xie, Boyi Zhang, Shaoli Hu, Chang Xu, Xiyu Ren, Yufei Li, Qiao Xiao, Huihao Jing, Kawai Chung, Wenbin Hu, Hongyu Luo, Lingyun Xie, Yisen Gao, Chunyang Li, Haoyu Huang, Wuganjin

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Multi Agent Systems](<https://devfeed.tech/topics/multi-agent-systems.md>), [Benchmark](<https://devfeed.tech/topics/benchmark.md>), [Computer vision](<https://devfeed.tech/topics/computer-vision.md>), [hierarchical agent systems](<https://devfeed.tech/topics/hierarchical-agent-systems.md>), [real-time](<https://devfeed.tech/topics/real-time.md>)

Tags: [adaptive](<https://devfeed.tech/tags/adaptive.md>), [agent](<https://devfeed.tech/tags/agent.md>), [agent-control](<https://devfeed.tech/tags/agent-control.md>), [agents](<https://devfeed.tech/tags/agents.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [benchmarking](<https://devfeed.tech/tags/benchmarking.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [camera](<https://devfeed.tech/tags/camera.md>), [control](<https://devfeed.tech/tags/control.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-ro](<https://devfeed.tech/tags/cs-ro.md>), [multi-agent](<https://devfeed.tech/tags/multi-agent.md>), [real-time](<https://devfeed.tech/tags/real-time.md>), [robot](<https://devfeed.tech/tags/robot.md>), [robot-arms](<https://devfeed.tech/tags/robot-arms.md>), [system](<https://devfeed.tech/tags/system.md>), [systems](<https://devfeed.tech/tags/systems.md>), [tasks](<https://devfeed.tech/tags/tasks.md>), [vision](<https://devfeed.tech/tags/vision.md>)

### AI overview

RoboTeam benchmarks agents controlling 4-16 robot arms across 36 long cooperative tasks, using camera images alone while simulated time continues during agent reasoning. In its adaptive setup, gpt-6.1-sol and opus-5.5 complete 28 and 27 tasks; four other tested models complete at most 3. Performance is costly and failure-prone: a typical gpt-6.1-sol task takes 3.5 hours and 218 million input tokens, with about a third of steps failing at least once and requiring retries. Four agent systems complete similar numbers of tasks but trade speed against token use: hierarchical and decentralized systems are faster and use up to 1.9 times the tokens, while single-agent uses fewer tokens but takes 1.6 times as long. The abstract concludes that no tested model or system is simultaneously fast, inexpensive, and reliable.

### Source excerpt

arXiv:2611.22138v1 Announce Type: cross Abstract: Multimodal agents can now control robot arms by calling tools and writing code. Existing benchmarks fall short in three ways: most test one or two arms on short tasks, many pause the world while the agent thinks, and many give it ground-truth object positions. We present RoboTeam, a simulated benchmark where agents control 4 to 16 robot arms in real time from camera images alone. Its 36 tasks are long jobs that the arms must do together, such as passing timbers or building a picture from hundreds of cubes. We start from the most flexible agent system, the adaptive system, where one agent decides when to fork delegates. With it, gpt-6.1-sol and opus-5.5 complete 28 and 27 tasks, and four other models at most 3. But at a high price: a typical task takes gpt-6.1-sol 3.5 hours and 218 million input tokens, and a third of its steps fail at least once and must be retried. We then compare it with three fixed systems: single-agent, hierarchical, and decentralized. All four complete a similar number of tasks but differ in time and tokens. The hierarchical and decentralized systems are faster but use up to 1.9 times the tokens, and the single agent uses fewer tokens but takes 1.6 times as long, so the adaptive system lies between them. Even so, no model or system is yet fast, cheap, and reliable at once. RoboTeam measures these limits where earlier benchmarks cannot: long multi-arm jobs, camera images only, and a world that never pauses.

## Segmentation-Guided Direct Capacity Estimation of Rooftop Photovoltaics from Aerial Imagery

DevFeed: [Segmentation-Guided Direct Capacity Estimation of Rooftop Photovoltaics from Aerial Imagery](<https://devfeed.tech/articles/segmentation-guided-direct-capacity-estimation-of-rooftop-photovoltaics-from-aerial-imagery-139957.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.22076>)

Author: Guibin Chen, Emma Fan, Yuzhang Lin

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Computer vision](<https://devfeed.tech/topics/computer-vision.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>)

Tags: [aerial-imagery](<https://devfeed.tech/tags/aerial-imagery.md>), [capacity](<https://devfeed.tech/tags/capacity.md>), [computer-vision](<https://devfeed.tech/tags/computer-vision.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [photovoltaics](<https://devfeed.tech/tags/photovoltaics.md>)

### AI overview

Direct Head estimates rooftop PV capacity from aerial imagery without deriving it as segmented area multiplied by power density. It retains U-Net segmentation and soft-mask pooling, then regresses nonnegative capacity from PV-region features and area descriptors. Across three runs on 7,590 French rooftop installations, including a department-based holdout, it lowered mean absolute error by 8.7% and 12.5% and symmetric mean absolute percentage error by 14.0% and 20.7% versus a parameter-matched area-density neural model. The reported comparisons indicate segmentation supplies useful spatial evidence while capacity need not be fixed by an area-density product.

### Source excerpt

arXiv:2611.22076v1 Announce Type: cross Abstract: Distribution planners need installation-level inventories of rooftop photovoltaic (PV) capacity, but interconnection records may be incomplete or spatially aggregated. Conventional image-based estimators segment PV arrays, convert the mask to physical area, and multiply by an estimated power density, coupling capacity prediction to mask-derived area. We propose Direct Head, a segmentation-guided architecture that retains U-Net segmentation and soft-mask pooling while directly regressing nonnegative capacity from PV-region features and area descriptors. We evaluate three independent runs on 7,590 French rooftop installations under the original split and a department-based holdout. Relative to a parameter-matched area-density neural model, Direct Head reduces mean absolute error by 8.7% and 12.5%, and symmetric mean absolute percentage error by 14.0% and 20.7%, respectively. Comparisons with conventional estimators, controlled architecture variants, auxiliary-loss ablations, and representative cases show that segmentation provides useful spatial evidence without requiring capacity to be determined by an area-density product.

## EgoHandWAM: Probing the Role of Hand Motion in Egocentric World Action Modeling

DevFeed: [EgoHandWAM: Probing the Role of Hand Motion in Egocentric World Action Modeling](<https://devfeed.tech/articles/egohandwam-probing-the-role-of-hand-motion-in-egocentric-world-action-modeling-139972.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.22151>)

Author: Kailin Wang, Haoxiang Jie, Yaoyuan Yan, Zhiyou Heng, Zhaosong Li

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Computer vision](<https://devfeed.tech/topics/computer-vision.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Physical AI](<https://devfeed.tech/topics/physical-ai.md>)

Tags: [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [hand](<https://devfeed.tech/tags/hand.md>), [modeling](<https://devfeed.tech/tags/modeling.md>), [motion](<https://devfeed.tech/tags/motion.md>), [probing](<https://devfeed.tech/tags/probing.md>)

### AI overview

EgoHandWAM predicts 3D wrist and fingertip trajectories and a future visual feature from a current egocentric image and recent hand states, trained on 751,065 EgoDex windows with a frozen DINOv2 encoder. On held-out development sessions, its action-only model lowers two-second wrist endpoint error from 60.26 to 56.48 mm versus a stationary forecast, but underestimates long reaches; future-feature supervision yields nearly identical trajectory accuracy. Recorded hand history improves visual-feature smooth L1 error by 1.32% over copying the current feature, while shuffling histories across sessions removes the gain. In a separate four-task probe, recorded future displacements plus tracking-validity masks reduce feature error by 3.93%; masks alone or a predicted trajectory add little. Replacing auxiliary robot examples with aligned human examples shows no consistent task-success increase on two simulated RoboTwin tasks across three seeds.

### Source excerpt

arXiv:2611.22151v1 Announce Type: cross Abstract: How much does hand motion tell us about what a first-person camera will see next? EgoHandWAM forecasts three-dimensional wrist and fingertip trajectories and a future visual feature from the current image and recent hand states. We train on 751,065 EgoDex windows using a frozen DINOv2 encoder. On held-out development sessions, the action-only model reduces two-second wrist endpoint error from 60.26 to 56.48~mm relative to a stationary forecast. It still underestimates long reaches. Adding future-feature supervision gives nearly identical trajectory accuracy. With the recorded hand history, the visual head reduces smooth L1 error by 1.32\% relative to copying the current feature. Shuffling histories across sessions erases this gain. In a separate four-task probe, adding recorded future displacements to observed context and future tracking-validity masks reduces visual-feature error by 3.93\%. Masks alone give little improvement. A trajectory predicted from the same observations adds little. On two simulated RoboTwin tasks, substituting aligned human examples for an auxiliary batch of robot examples gives no consistent increase in task success across three training seeds.

## CueClash: Frontier Vision-Language Models Judge 3-D Relations Close to the Picture Plane

DevFeed: [CueClash: Frontier Vision-Language Models Judge 3-D Relations Close to the Picture Plane](<https://devfeed.tech/articles/cueclash-frontier-vision-language-models-judge-3-d-relations-close-to-the-picture-plane-139980.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.22180>)

Author: Soham Dan

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [vlm](<https://devfeed.tech/topics/vlm.md>), [Computer vision](<https://devfeed.tech/topics/computer-vision.md>), [3D](<https://devfeed.tech/topics/3d.md>), [Claude](<https://devfeed.tech/topics/claude.md>), [Frontier AI](<https://devfeed.tech/topics/frontier-ai.md>)

Tags: [3-d](<https://devfeed.tech/tags/3-d.md>), [bench](<https://devfeed.tech/tags/bench.md>), [claude-opus-5-5](<https://devfeed.tech/tags/claude-opus-5-5.md>), [computer-vision](<https://devfeed.tech/tags/computer-vision.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-ro](<https://devfeed.tech/tags/cs-ro.md>), [gemini-3-6-flash](<https://devfeed.tech/tags/gemini-3-6-flash.md>), [gpt-5-5](<https://devfeed.tech/tags/gpt-5-5.md>), [maps](<https://devfeed.tech/tags/maps.md>), [open](<https://devfeed.tech/tags/open.md>), [vision-language-models](<https://devfeed.tech/tags/vision-language-models.md>)

### AI overview

CueClash tests whether VLMs judge 3-D relations from scene layout or rely on 2-D picture cues. In rendered scenes, Claude Opus 5.5, GPT-5.5 and Gemini 3.6 Flash show an 18-37 percentage-point accuracy gap between statements where the cue is right and wrong; excluding two relations with confounds reduces it to 7-29 points. With ground coordinates and relation definitions, accuracy reaches 99-100%, locating the difficulty in inferring layout from the image. Overhead maps generally preserve depth order but shorten depth: GPT-5.5 largely copies the picture, while Opus 5.5 and Gemini undo about 0.6 of the shortening; Gemini then applies further uniform depth shrinkage. Pixel measurements with camera depths remove much of the gap for objects fitting through openings. The three models also lose accuracy on CV-Bench photos when a reliable 2-D cue misleads; two open models pass layout information to the language model, but their maps lose much of it.

### Source excerpt

arXiv:2611.22180v1 Announce Type: cross Abstract: A vision-language model (VLM) can judge 3-D relations from the picture or from the ground layout. In photographs the two usually agree. We separate them with CueClash, rendered scenes in which the 2-D cue for each of five relations is right on exactly half the statements. The shortcut gap is accuracy when the cue is right minus accuracy when it is wrong. For Claude Opus 5.5, GPT-5.5 and Gemini 3.6 Flash at low reasoning effort it is +18 to +37 percentage points. It is +7 to +29 without two relations whose cue-wrong items are confounded with borderline cases or occlusion. Given ground coordinates and each relation's definition, they answer 99 to 100% correctly. Pixel measurements with camera depths remove most of the gap for objects fitting through openings. The failure lies in the step from the picture to the layout. The maps that the models draw of the scene from above mostly keep the depth order but, like the picture, shorten depth. GPT-5.5's maps stay close to a copy of the picture. Opus 5.5's and Gemini's maps undo about 0.6 of this shortening. Gemini's then shrink all depths by a further uniform factor. Opus 5.5 follows the picture mainly on fits, and its maps correct depth about as much as human observers of real scenes do. The three models also lose accuracy on CV-Bench photographs where a reliable 2-D cue misleads. In two open models the layout information reaches the language model, but their maps lose much of it.

## Reading Scene States: Uncovering Safety Failures in Vision-Language Models

DevFeed: [Reading Scene States: Uncovering Safety Failures in Vision-Language Models](<https://devfeed.tech/articles/reading-scene-states-uncovering-safety-failures-in-vision-language-models-140133.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.23206>)

Author: Tongxi Wu, Yuyang Cheng, Jiacheng Xian, Menglin Yang, Kai Chen, Jian Zhang, Yang Gao

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [LLM security](<https://devfeed.tech/topics/llm-security.md>), [vlm](<https://devfeed.tech/topics/vlm.md>), [Computer vision](<https://devfeed.tech/topics/computer-vision.md>)

Tags: [analysis](<https://devfeed.tech/tags/analysis.md>), [attack](<https://devfeed.tech/tags/attack.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [failures](<https://devfeed.tech/tags/failures.md>), [image](<https://devfeed.tech/tags/image.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [moderation](<https://devfeed.tech/tags/moderation.md>), [reasoning](<https://devfeed.tech/tags/reasoning.md>), [safety](<https://devfeed.tech/tags/safety.md>), [vision-language-models](<https://devfeed.tech/tags/vision-language-models.md>), [vulnerability](<https://devfeed.tech/tags/vulnerability.md>)

### AI overview

The abstract reports that vision-language models can give harmful assistance by interpreting visual events even when users do not explicitly state the harmful goal. SceneSafetyBench contains 375 image-question pairs covering three scene states and five visual presentations; across eight models, these inputs produced attack success rates of 45.6-72.3%. The proposed S3TA attack translates harmful goals into scene states, queries image-question pairs independently, and assembles responses. Across seven MM-SafetyBench categories and eight open-weight and commercial models, its average success rate was 71.2%, 6.1 percentage points above the strongest baseline. The authors also report limited protection from text-side filters and visual moderation. These findings are based on the abstract provided.

### Source excerpt

arXiv:2611.23206v1 Announce Type: cross Abstract: Vision-language models (VLMs) can provide harmful assistance by interpreting visual events, even without an explicit request for the underlying harmful objective. We identify this \textbf{scene-state safety vulnerability} and introduce \textbf{SceneSafetyBench}, comprising 375 image--question pairs spanning Initial, Medium, and Ultimate scene states and five visual presentations. Across eight VLMs, scene-state inputs elicit harmful assistance with attack success rates (ASRs) of 45.6--72.3\%. Comparative analysis across three VLMs shows that scene-state inputs most closely resemble QR-related images in their layer-wise safety-related signal profiles, while exhibiting lower average late-layer readouts and distinct signal dynamics during generation. To turn this vulnerability into a systematic attack, we propose \textbf{S3TA}, which translates harmful objectives into scene states, queries image--question pairs independently, and assembles the responses. On seven MM-SafetyBench categories across eight open-weight and commercial VLMs, S3TA achieves a model-average ASR of \textbf{71.2\%, exceeding the strongest baseline by 6.1 percentage points}. Text-side filters and visual moderation provide limited protection. These findings highlight the need to safeguard the assistance models infer through scene-grounded reasoning, beyond screening input content.

## The Flip Test: Video World Models Fail to Learn the World State

DevFeed: [The Flip Test: Video World Models Fail to Learn the World State](<https://devfeed.tech/articles/the-flip-test-video-world-models-fail-to-learn-the-world-state-137499.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.19822>)

Author: Mi Luo, Lijun Yu, Adams Wei Yu, Ang Cao, Baochen Sun, Ming-Hsuan Yang, Nan Hua, Yuan Hao, Alex Dimakis, Ya Xu, Ed H. Chi, Xuezhi Wang, Yinxiao Liu

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [World models](<https://devfeed.tech/topics/world-models.md>), [Computer vision](<https://devfeed.tech/topics/computer-vision.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>)

Tags: [computer-vision](<https://devfeed.tech/tags/computer-vision.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-ro](<https://devfeed.tech/tags/cs-ro.md>), [prediction](<https://devfeed.tech/tags/prediction.md>), [state](<https://devfeed.tech/tags/state.md>), [test](<https://devfeed.tech/tags/test.md>), [video](<https://devfeed.tech/tags/video.md>), [world-models](<https://devfeed.tech/tags/world-models.md>)

### AI overview

FlipTest finds that the 11 evaluated video world models do not reliably track small changes that alter a scene's future. Across 1,680 initial frames from 10 simulated and 15 real-world tasks, the best model scores 18.3% on minimal-pair accuracy, below the 25% random-guessing rate. Models also flip predictions just as often when the future stays the same as when it changes, and show a flat response across dense sweeps. On the same frames, a vision-language model responds selectively near the boundary, while a coding agent measuring the image comes close to locating it. The authors argue that plausible next-frame prediction is insufficient: world models must identify the causal states governing future dynamics. The supplied text is an abstract, so it does not establish coverage of the full paper.

### Source excerpt

arXiv:2611.19822v1 Announce Type: cross Abstract: Video world models promise to predict the future of a scene from its present. Do current models keep this promise? We probe this question counterfactually. When a small change to the present flips the future, such as a box resting a few millimeters before or past a table edge, a model that tracks the state of the world must flip its prediction too. We build FlipTest, a three-level hierarchy of increasingly strict tests over 1,680 initial frames from 10 simulated and 15 real-world tasks. At the base, visual minimal pairs, two nearly identical frames whose true futures differ, test whether predictions change across the boundary; matched controls go further and test whether they change only where the future does; and dense sweeps, the strictest level, locate exactly where they change. All 11 video world models we evaluate fall short. The best averages 18.3% pair accuracy over real and simulated scenes, below the 25% of random guessing. Models flip as often on steps that leave the future unchanged as on steps that change it, and their response stays flat across the sweeps. On the same frames, a vision-language model is selective and traces a near S-shaped curve near the boundary, and a coding agent that measures the image comes close to the true boundary. This implies that the deciding information is in the frame, and video models just miss it. We therefore urge the community to rethink world modeling as pure video prediction. Next-frame prediction must do more than draw plausible pixels; it must also identify the causal states that control future dynamics.

## Hard or Mislabeled? Failure Screens Enrich Vision Benchmarks for Label Errors

DevFeed: [Hard or Mislabeled? Failure Screens Enrich Vision Benchmarks for Label Errors](<https://devfeed.tech/articles/hard-or-mislabeled-failure-screens-enrich-vision-benchmarks-for-label-errors-137384.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.19044>)

Author: Haolin Guo

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Computer vision](<https://devfeed.tech/topics/computer-vision.md>), [ImageNet](<https://devfeed.tech/topics/imagenet.md>), [cifar-10](<https://devfeed.tech/topics/cifar-10.md>)

Tags: [audit](<https://devfeed.tech/tags/audit.md>), [audits](<https://devfeed.tech/tags/audits.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [cifar-10](<https://devfeed.tech/tags/cifar-10.md>), [classifier](<https://devfeed.tech/tags/classifier.md>), [classifiers](<https://devfeed.tech/tags/classifiers.md>), [computer-vision](<https://devfeed.tech/tags/computer-vision.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [enrichment](<https://devfeed.tech/tags/enrichment.md>), [errors](<https://devfeed.tech/tags/errors.md>), [failure](<https://devfeed.tech/tags/failure.md>), [screens](<https://devfeed.tech/tags/screens.md>), [vision](<https://devfeed.tech/tags/vision.md>)

### AI overview

Failure screens for hard vision benchmarks can disproportionately select mislabeled images. On ImageNet's twice-relabeled validation set, 61% of the 1,892 images missed by all 66 supervised classifiers had wrong labels, versus 11.1% overall. As screens become stricter, the mislabeled share generally rises along a curve that predicted held-out model-family screens with a 2.4 percentage-point mean error; replays of ImageNet-D and ImageNet-Hard were within 0.1 and 6.3 points. The curve did not transfer uniformly: strict screens on ImageNet-V2 and CIFAR-10 showed stronger enrichment than predicted, while ImageNet-A's screen, which restricts choices to 200 of 1,000 classes, fell below it. Human relabeling found 67% of the all-66-failure ImageNet-V2 images mislabeled, compared with 46% predicted. The abstract also reports that 32% of ImageNet-Hard's ImageNet images have no correct label by ReImageNet. ScreenAudit and corrected answer sets are described as planned releases.

### Source excerpt

arXiv:2611.19044v1 Announce Type: cross Abstract: Hard vision benchmarks often keep images that strong classifiers get wrong. Since a classifier is also marked wrong when the label is wrong, such failure screens may collect label errors. We measure this on ImageNet's twice-relabeled validation set with 103 classifiers. Among the 1,892 images that all 66 supervised classifiers get wrong, 61% have a wrong label, versus 11.1% overall. The fewer images a screen lets through, the more of them are mislabeled, along approximately one curve. Fit on one model family, this curve predicts the screens of two held-out model families with a mean error of 2.4 percentage points, and replays of ImageNet-D's and ImageNet-Hard's screens within 0.1 and 6.3 points. On new pools, ImageNet-V2 and CIFAR-10, enrichment at the strictest screens is stronger than the curve predicts: on ImageNet-V2, by human relabeling, 67% of the images that all 66 classifiers get wrong are mislabeled, against 46% predicted. Screens that let models choose among only some of the classes, as ImageNet-A's does (200 of 1,000), fall below the curve. By ReImageNet, 32% of ImageNet-Hard's ImageNet images have no correct label. ScreenAudit, which comprises an audit tool, a script that rebuilds corrected answer sets for the 5,464 decided ImageNet-Hard images (of 5,646) from ReImageNet's gated release, and per-image audits of 1,198 benchmark images, will be released later.

## Remember, Replay, Adapt: Learning from Past Experience in Online Vision-and-Language Navigation

DevFeed: [Remember, Replay, Adapt: Learning from Past Experience in Online Vision-and-Language Navigation](<https://devfeed.tech/articles/remember-replay-adapt-learning-from-past-experience-in-online-vision-and-language-navigation-137511.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.19901>)

Author: Chenhao Ding, Jizhou Han, Lixin Xiu, Muyao Wang, Songlin Dong, Hideki Nakayama

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Robot Navigation](<https://devfeed.tech/topics/robot-navigation.md>), [Computer vision](<https://devfeed.tech/topics/computer-vision.md>), [generalization in machine learning](<https://devfeed.tech/topics/generalization-in-machine-learning.md>)

Tags: [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [episodic-memory](<https://devfeed.tech/tags/episodic-memory.md>), [experience](<https://devfeed.tech/tags/experience.md>), [feedback](<https://devfeed.tech/tags/feedback.md>), [framework](<https://devfeed.tech/tags/framework.md>), [learning](<https://devfeed.tech/tags/learning.md>), [memory](<https://devfeed.tech/tags/memory.md>), [navigation](<https://devfeed.tech/tags/navigation.md>), [online](<https://devfeed.tech/tags/online.md>), [performance](<https://devfeed.tech/tags/performance.md>), [policy](<https://devfeed.tech/tags/policy.md>), [reuse](<https://devfeed.tech/tags/reuse.md>), [stable](<https://devfeed.tech/tags/stable.md>), [vision](<https://devfeed.tech/tags/vision.md>)

### AI overview

R2A adapts vision-and-language navigation policies by reusing feedback from both successful and failed past episodes. Its Success-Failure Memory retrieves relevant episodes using incoming instructions; Prior-Anchored Adaptation reinforces successful trajectories and suppresses failures relative to a frozen pretrained policy while regularizing behavioral drift. The approach is designed to work without source-domain data or expert supervision. The abstract reports consistent gains across REVERIE, R2R, and R2R-CE with five backbones; on REVERIE's validation unseen split, DUET's success rate rises from 46.98% to 69.84% and SPL from 33.73% to 50.98%.

### Source excerpt

arXiv:2611.19901v1 Announce Type: cross Abstract: Existing feedback-based methods for online vision-and-language navigation (VLN) primarily learn from current episode outcomes, leaving previously acquired feedback underutilized. Inspired by human episodic memory, we propose R2A, an online VLN adaptation framework that continually reuses successful and failed navigation experiences for policy learning. Specifically, Success-Failure Memory (SFM) retains successful and failed navigation episodes with acquired feedback and retrieves task-relevant experiences based on incoming instructions, enabling historical feedback to guide subsequent navigation. Prior-Anchored Adaptation (PAA) learns from these experiences by reinforcing successful trajectories and suppressing failed ones relative to a frozen pretrained policy, while regularizing behavioral drift. Together, these modules enable continual feedback reuse and stable policy adaptation without source-domain data or expert supervision. Extensive experiments across three benchmarks, REVERIE, R2R, and R2R-CE, and five navigation backbones demonstrate consistent performance gains. On the REVERIE validation unseen split, R2A improves DUET's SR from 46.98% to 69.84% and SPL from 33.73% to 50.98%.

## Lesion-Aware Explainable AI for Diabetic Retinopathy Grading from Fundus Images

DevFeed: [Lesion-Aware Explainable AI for Diabetic Retinopathy Grading from Fundus Images](<https://devfeed.tech/articles/lesion-aware-explainable-ai-for-diabetic-retinopathy-grading-from-fundus-images-137462.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.19550>)

Author: Truong Thinh Nguyen, Hai Phong Nguyen, Truong Thanh Hung Nguyen, Giang Son Tran, Hung Cao

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Computer vision](<https://devfeed.tech/topics/computer-vision.md>), [Deep neural networks](<https://devfeed.tech/topics/deep-neural-networks.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [Image](<https://devfeed.tech/topics/image.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [computer-vision](<https://devfeed.tech/tags/computer-vision.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [deep-learning](<https://devfeed.tech/tags/deep-learning.md>), [dr](<https://devfeed.tech/tags/dr.md>), [explainable-ai](<https://devfeed.tech/tags/explainable-ai.md>), [fusion](<https://devfeed.tech/tags/fusion.md>), [images](<https://devfeed.tech/tags/images.md>), [reliability](<https://devfeed.tech/tags/reliability.md>)

### AI overview

The proposed Consensus-Calibrated Explanation Map (CCEM) combines Grad-CAM++, Ada-SISE, and SmoothGrad Integrated Gradients with image-specific weights from annotation-free reliability cues, aiming to make DR-grading explanations more lesion-supported and faithful to the model. It explains a frozen EfficientNet-B7 ordinal grader. On 193 MESSIDOR images with expert lesion annotations, CCEM achieved a 69.61% Soft Pointing Game score (a relative saliency measure, not a lesion detection rate), a 0.570 deletion AUC, and a 0.115 deletion-insertion balance; each result was significantly better than the strongest single explainer on paired tests. The grader's reported QWK was 0.920 on the hidden Kaggle test set and 0.559 on external MAPLES-DR. The excerpt does not provide further methodological or evaluation details.

### Source excerpt

arXiv:2611.19550v1 Announce Type: cross Abstract: Post-hoc explanations of deep-learning models for diabetic retinopathy (DR) grading are difficult to trust because different attribution methods highlight different retinal regions for the same image, and a visually plausible heatmap does not show that the highlighted pixels drive the prediction. We propose the Consensus-Calibrated Explanation Map (CCEM), which fuses Grad-CAM++, Ada-SISE, and SmoothGrad Integrated Gradients (SG-IG) using image-specific weights derived from annotation-free reliability cues: deletion sensitivity, inter-method agreement, compactness, retinal containment, and peak structure. CCEM explains a frozen ordinal grader, an EfficientNet-B7 with CBAM and a scalar regression head, pre-trained on EyePACS 2015 in Stage 1 and fine-tuned on APTOS 2019 in Stage 2 with quadratic weighted kappa (QWK)-optimized thresholds; it reaches a QWK of 0.920 on the hidden Kaggle test set and 0.559 on the external MAPLES-DR cohort. Explanations are evaluated on MAPLES-DR, 198 MESSIDOR images with expert pixel-level lesion annotations that are never used for training, threshold fitting, or fusion. On the 193 images with annotated lesions, CCEM obtains the highest Soft Pointing Game score (69.61%; 95% CI 63.7-75.3), defined as the ratio of the strongest lesion-supported saliency to the global map maximum rather than a lesion detection rate, together with the lowest deletion AUC (0.570) and the best deletion-insertion balance (0.115); each gain is significant against the strongest single explainer on paired tests. Annotation-free, per-image reliability weighting therefore turns three disagreeing explainers into one map whose peak evidence is more lesion-supported and more faithful to the model than that of any single explainer.

## TyphoonWorld: A Residual Token World Model for Probabilistic Forecasting of Typhoon Imagery

DevFeed: [TyphoonWorld: A Residual Token World Model for Probabilistic Forecasting of Typhoon Imagery](<https://devfeed.tech/articles/typhoonworld-a-residual-token-world-model-for-probabilistic-forecasting-of-typhoon-imagery-138781.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.18740>)

Author: Xiaoyu Chen, Tianyi Li, Xuanyi Zhang

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [World models](<https://devfeed.tech/topics/world-models.md>), [Computer vision](<https://devfeed.tech/topics/computer-vision.md>)

Tags: [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [forecasting](<https://devfeed.tech/tags/forecasting.md>), [imagery](<https://devfeed.tech/tags/imagery.md>), [model](<https://devfeed.tech/tags/model.md>), [probabilistic](<https://devfeed.tech/tags/probabilistic.md>), [probabilistic-forecasting](<https://devfeed.tech/tags/probabilistic-forecasting.md>), [residual](<https://devfeed.tech/tags/residual.md>), [token](<https://devfeed.tech/tags/token.md>), [world](<https://devfeed.tech/tags/world.md>), [world-model](<https://devfeed.tech/tags/world-model.md>)

### AI overview

TyphoonWorld generates probabilistic typhoon imagery by sampling discrete-token residual fields conditioned on a regression forecast, recent frames, and best-track state, then adding those residuals to the regression forecast. Across 276 windows from 46 held-out Digital Typhoon storms, it matched the regression model's squared error and reduced CRPS for the area colder than 220 K by 14-39% versus each tested ensemble. Its ensemble spread tracked forecast error with a reported correlation of 0.78, matching the Gaussian network on that measure. It used 56% of the parameters of the unconditional token world model. These findings are from the supplied abstract.

### Source excerpt

arXiv:2611.18740v1 Announce Type: cross Abstract: A satellite forecast of a typhoon should give a low-error picture of the next two days, together with an ensemble whose members are coherent fields and whose spread responds to the storm at hand. A small direct regression network already gives the lowest squared error on held-out storms, so a generative model has to earn its cost through the part of the future that the regression smooths away. We present TyphoonWorld, a discrete-token world model that generates only that part. Conditioned on tokens of the regression forecast, the latest frames and the best-track state at forecast time, it samples residual fields lead by lead, and members add these residuals to the regression center. We compare it under one protocol with per-pixel Gaussian networks, a diffusion model of the same residuals, residuals from other storms and an unconditional token world model, on 276 forecast windows from 46 held-out storms of Digital Typhoon. TyphoonWorld matches the squared error of the regression, lowers the CRPS of the area colder than 220 K, a direct measure of deep convection, by 14 to 39% against every other ensemble, and ties its spread to forecast error as closely as the Gaussian network, with a correlation of 0.78 across windows. It uses 56% of the parameters of the unconditional world model.

## Unreasonable Confidence from Simulator Misspecification: Directional Robustness on the SPEED+ Sim-to-Real Benchmark

DevFeed: [Unreasonable Confidence from Simulator Misspecification: Directional Robustness on the SPEED+ Sim-to-Real Benchmark](<https://devfeed.tech/articles/unreasonable-confidence-from-simulator-misspecification-directional-robustness-on-the-speed-sim-to-real-benchmark-137561.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.20321>)

Author: Nikolay Dolgov

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Computer vision](<https://devfeed.tech/topics/computer-vision.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Benchmark](<https://devfeed.tech/topics/benchmark.md>)

Tags: [adapting](<https://devfeed.tech/tags/adapting.md>), [aluminium](<https://devfeed.tech/tags/aluminium.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [code](<https://devfeed.tech/tags/code.md>), [confidence](<https://devfeed.tech/tags/confidence.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [data](<https://devfeed.tech/tags/data.md>), [domain](<https://devfeed.tech/tags/domain.md>), [domains](<https://devfeed.tech/tags/domains.md>), [error](<https://devfeed.tech/tags/error.md>), [errors](<https://devfeed.tech/tags/errors.md>), [estimator](<https://devfeed.tech/tags/estimator.md>), [experiment](<https://devfeed.tech/tags/experiment.md>), [features](<https://devfeed.tech/tags/features.md>), [flight](<https://devfeed.tech/tags/flight.md>), [gap](<https://devfeed.tech/tags/gap.md>), [robustness](<https://devfeed.tech/tags/robustness.md>), [sim-to-real](<https://devfeed.tech/tags/sim-to-real.md>), [simulator](<https://devfeed.tech/tags/simulator.md>)

### AI overview

On SPEED+, a simple monocular pose estimator trained on synthetic images can be confidently wrong on real imagery: regions calibrated for 90% coverage on synthetic data covered only 9-15% of rotation errors on the harshly lit sunlamp domain and 21-38% on lightbox. In a 2x2 study, scaling data and features reduced synthetic rotation error from 1.65 to 1.03 rad but improved sunlamp by only 0.05 rad, while adapting the backbone's last stage transferred 63-83% of its additional synthetic gain to both real domains. Photometric augmentation was directional: after backbone adaptation it improved lightbox error from 1.17 to 0.99 rad on every seed, while sunlamp changes stayed within seed variation; with a frozen backbone it helped sunlamp more. The simulator's radiometric mismatch includes missing highlights, hard shadows, and sensor saturation. The supplied text is an abstract, so it does not establish further methodological details.

### Source excerpt

arXiv:2611.20321v1 Announce Type: cross Abstract: Vision-based navigation systems for orbit must be trained almost entirely on synthetic imagery, because real labelled images of a given spacecraft in flight rarely exist. Benchmarks like SPEED+ are built for that situation: the training set comes from a simulator, and the test sets are real images of a satellite mockup under two kinds of lighting. That simulator approximates how light reflects off mylar and aluminium, and like every scientific simulator it is misspecified. Its error is radiometric, since it misses the specular highlights, hard shadows and sensor saturation common in real images. We probe the resulting synthetic-to-real gap, the central risk for such systems, with a deliberately simple monocular pose estimator. In a 2x2 study of backbone adaptation and photometric augmentation, trained on synthetic images and tested on both real hardware-in-the-loop domains, we find three things. (1) The model does not know it is failing: regions conformally calibrated to 90% on synthetic data cover only 9-15% of rotation errors on sunlamp and 21-38% on lightbox, so a wrong pose carries the confidence of a correct one. (2) Transfer is asymmetric: scaling data and features improved synthetic rotation error from 1.65 to 1.03 rad but moved the harshly lit sunlamp domain by only 0.05 rad, under twice its seed s.d., whereas adapting the backbone's last stage carried 63-83% of its extra synthetic gain to both real domains. (3) Once the backbone adapts, augmentation helps only in the direction it models: our photometric stack improved the diffusely lit lightbox domain from 1.17 to 0.99 rad on every seed but moved sunlamp no more than seed-to-seed variation, while with the backbone frozen it helped sunlamp more. Every post-review experiment was pre-registered, and the code, per-image predictions and pre-registration record are at https://github.com/lokindoo/spacecraft-pose-estimation.

## From Images to Grounded Edits: A Real-World Benchmark for Chinese Grammatical Error Correction in Handwritten Essays

DevFeed: [From Images to Grounded Edits: A Real-World Benchmark for Chinese Grammatical Error Correction in Handwritten Essays](<https://devfeed.tech/articles/from-images-to-grounded-edits-a-real-world-benchmark-for-chinese-grammatical-error-correction-in-handwritten-essays-135156.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.17966>)

Author: Weiru Zhang, Yuzhong Hong, Junwei Bao, Lemao Liu, Dejing Dou

Published: 2026-11-16T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Computer vision](<https://devfeed.tech/topics/computer-vision.md>), [Benchmark](<https://devfeed.tech/topics/benchmark.md>), [object-detection](<https://devfeed.tech/topics/object-detection.md>), [Multi Agent Systems](<https://devfeed.tech/topics/multi-agent-systems.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [agentic](<https://devfeed.tech/tags/agentic.md>), [ai-assisted](<https://devfeed.tech/tags/ai-assisted.md>), [analysis](<https://devfeed.tech/tags/analysis.md>), [annotations](<https://devfeed.tech/tags/annotations.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [detection](<https://devfeed.tech/tags/detection.md>), [digital](<https://devfeed.tech/tags/digital.md>), [discovery](<https://devfeed.tech/tags/discovery.md>), [education](<https://devfeed.tech/tags/education.md>), [error-correction](<https://devfeed.tech/tags/error-correction.md>), [essay](<https://devfeed.tech/tags/essay.md>), [essays](<https://devfeed.tech/tags/essays.md>), [eval](<https://devfeed.tech/tags/eval.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [grounded](<https://devfeed.tech/tags/grounded.md>), [grounding](<https://devfeed.tech/tags/grounding.md>), [images](<https://devfeed.tech/tags/images.md>), [real-world](<https://devfeed.tech/tags/real-world.md>)

### AI overview

The benchmark reframes Chinese grammatical error correction for handwritten essays as an image-to-grounded-edit task: systems must identify corrections and locate them on essay images. G-CGEC-Eval contains real K-12 Chinese essay images with fine-grained annotations and uses adapted IoU metrics alongside a more tolerant center-distance measure. In the reported evaluations, direct single-pass vision-language models performed poorly, while pipelines and agentic workflows using OCR and character grounding improved results. An error-type-based multi-agent workflow performed best among the tested approaches. The task remains unsolved, with linguistic error discovery and character-level grounding identified as key bottlenecks.

### Source excerpt

arXiv:2611.17966v1 Announce Type: cross Abstract: Chinese Grammatical Error Correction (CGEC) is typically formulated as a text-to-text problem. While suitable for digital texts, this formulation falls short in real-world K-12 education, where AI-assisted corrections are expected directly on essay images, which are primarily handwritten and exhibit many irregularities. To bridge this gap, we introduce Grounded Chinese Grammatical Error Correction (G-CGEC), an image-to-grounded-edit task requiring systems to predict both corrections and their positions on essay images. We propose G-CGEC-Eval, a benchmark comprising real-world images of K-12 Chinese essays with fine-grained annotations. For evaluation, we adapt IoU-based metrics from object detection and propose a more tolerant alternative based on center-distance matching. Using this benchmark, we evaluate open-source and proprietary vision-language models under a direct single-pass setting, as well as settings with auxiliary components (OCR and character grounding) coordinated via pipelining or agentic harnessing. Results show that even top-tier proprietary models perform surprisingly poorly in the single-pass setting, while pipelining and agentic harnessing yield meaningful improvements. Notably, an error-type-based multi-agent workflow achieves the best performance, establishing it as the strongest baseline for future G-CGEC research. Through ablations and fine-grained analysis, we find the task remains far from solved, with linguistic error discovery and character-level grounding as the primary bottlenecks. This underscores the need for models that better integrate visual perception with linguistic reasoning and visual grounding.

## TelePhysica: Agentic Reconstruction of Physically Interactive Scenes

DevFeed: [TelePhysica: Agentic Reconstruction of Physically Interactive Scenes](<https://devfeed.tech/articles/telephysica-agentic-reconstruction-of-physically-interactive-scenes-135163.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.18078>)

Author: Ruiqi Shu, Xufeng Zhan, Yuanzhi Liang, Haibin Huang, Chi Zhang, Xuelong Li

Published: 2026-11-16T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Computer vision](<https://devfeed.tech/topics/computer-vision.md>), [3D](<https://devfeed.tech/topics/3d.md>), [animations](<https://devfeed.tech/topics/animations.md>), [AI Chat](<https://devfeed.tech/topics/ai-chat.md>)

Tags: [2](<https://devfeed.tech/tags/2.md>), [3d](<https://devfeed.tech/tags/3d.md>), [3d-reconstruction](<https://devfeed.tech/tags/3d-reconstruction.md>), [abstract](<https://devfeed.tech/tags/abstract.md>), [agent](<https://devfeed.tech/tags/agent.md>), [agentic](<https://devfeed.tech/tags/agentic.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [coding](<https://devfeed.tech/tags/coding.md>), [coding-agent](<https://devfeed.tech/tags/coding-agent.md>), [condition](<https://devfeed.tech/tags/condition.md>), [control](<https://devfeed.tech/tags/control.md>), [coupling](<https://devfeed.tech/tags/coupling.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [declarative](<https://devfeed.tech/tags/declarative.md>), [diffusion](<https://devfeed.tech/tags/diffusion.md>), [diffusion-models](<https://devfeed.tech/tags/diffusion-models.md>), [drive](<https://devfeed.tech/tags/drive.md>), [element](<https://devfeed.tech/tags/element.md>), [execution](<https://devfeed.tech/tags/execution.md>), [interactive](<https://devfeed.tech/tags/interactive.md>)

### AI overview

TelePhysica reconstructs a physically executable 3D world from one photograph, then uses simulated motion to guide video generation. A vision-language coding agent creates visual meshes and physical models, checked through silhouette critique and a zero-force gravity-equilibrium preflight. The verified scene is simulated with a solver for rigid bodies, articulated chains, cloth, and tetrahedral elastic materials; its trajectories condition a pretrained video diffusion model. On a 100-scene benchmark, equilibrium-verified recovery rose from 0% for single-pass synthesis to 81%. Simulated-trajectory conditioning yielded 2-5x greater dynamic motion-mask overlap and preserved motion direction under reversed forces where direct image-to-video models failed. In cloth ablations, finite-element models showed over ten times the localized corner displacement of matched rigid-plate proxies, along with out-of-plane pleating.

### Source excerpt

arXiv:2611.18078v1 Announce Type: cross Abstract: Direct image-to-video diffusion models entangle pixel synthesis with implicit latent motion, precluding persistent 3D mechanical equilibrium, mid-rollout intervention, and counterfactual force control. Conversely, single-pass programmatic 3D reconstruction from an uncalibrated photograph leaves monocular depth and support clearance underconstrained, causing reconstructed scenes to interpenetrate or collapse under gravity. We present TelePhysica, a closed-loop framework that reconstructs an executable, multi-regime physical world from a single photograph to drive stateful 3D interaction and trajectory-grounded video synthesis. A vision-language coding agent synthesizes separable visual meshes and declarative physical models within a two-stage verification loop that pairs pixel-aligned silhouette critique with a zero-force gravitational equilibrium preflight. The verified world compiles into a unified solver coupling rigid bodies, articulated chains, co-dimensional cloth shells, and 3D tetrahedral elastodynamics, whose simulated rollouts condition pretrained video diffusion models through pixel-aligned structural streams. On a 100-scene single-image benchmark, closed-loop execution raises equilibrium-verified world recovery from $0\%$ in single-pass synthesis to $81\%$. Controlled constitutive ablations show over an order-of-magnitude larger localized corner displacement and out-of-plane pleating in finite-element cloth than in matched rigid-plate proxies. Conditioning video diffusion on simulated trajectories increases dynamic motion-mask overlap by $2{\times}\text{--}5{\times}$ and preserves motion direction under counterfactual force reversals where direct image-to-video models fail.

## OmniScore-Bench: Evaluating Vision-Language Models on Musical Score Reading

DevFeed: [OmniScore-Bench: Evaluating Vision-Language Models on Musical Score Reading](<https://devfeed.tech/articles/omniscore-bench-evaluating-vision-language-models-on-musical-score-reading-135134.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.17870>)

Author: Congren Dai, Zhancheng Guo, Danni Zhao, Wenchu Wang, Xiaobing Li, Maosong Sun

Published: 2026-11-16T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [LLM evaluation / benchmarking](<https://devfeed.tech/topics/llm-evaluation-benchmarking.md>), [vlm](<https://devfeed.tech/topics/vlm.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Computer vision](<https://devfeed.tech/topics/computer-vision.md>)

Tags: [bench](<https://devfeed.tech/tags/bench.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [optical](<https://devfeed.tech/tags/optical.md>), [score](<https://devfeed.tech/tags/score.md>), [transcription](<https://devfeed.tech/tags/transcription.md>), [vision-language-models](<https://devfeed.tech/tags/vision-language-models.md>)

### AI overview

OmniScore-Bench evaluates vision-language models' musical score reading across 3,000 questions, six aspects and 24 task subtypes. Questions define visible scope, reference annotations and answer schemas, enabling automatic evaluation from symbol localisation to musical consistency and evidence-linked question answering. Tests of 12 models found operation-specific capability profiles and different transcription leaders for excerpt, single-page and multipage tasks. Controlled analyses examined score images, auxiliary givens and additional page context; on a tested subset, removing images exposed visual dependence in pitch and repeat interpretation while measuring performance from public context alone. Separate format and content analyses distinguish answer-format violations from musical errors.

### Source excerpt

arXiv:2611.17870v1 Announce Type: new Abstract: Vision-language models (VLMs) can answer questions about musical scores, but their ability to recognise notation, recover its structure and interpret its musical meaning requires separate evaluation. Existing optical music recognition (OMR) resources establish detection and transcription measurements; a transcription score alone leaves the strengths and bottlenecks of a VLM score reader unresolved. We introduce OmniScore-Bench, an evaluation of OMR capabilities in VLMs through 3,000 questions spanning six aspects and 24 task subtypes. Each question specifies a visible scope, reference annotations and an answer schema, enabling automatic evaluation from symbol localisation to musical consistency and evidence-linked question answering. Our assessment of twelve VLMs reveals operation-specific capability profiles and different transcription leaders across excerpt, single-page and multipage tasks. Controlled analyses examine the roles of score images, auxiliary givens and additional page context. On the tested subset, image removal exposes visual dependence in pitch and repeat interpretation while measuring how much performance remains with public context alone. Separate format and content analyses distinguish answer-format violations from musical errors. OmniScore-Bench supports diagnosing which score-reading operations VLMs perform reliably and which require improvement.

## X-Nav Technical Report: Real-World Long-Horizon Indoor-to-Outdoor Navigation on a Humanoid Robot

DevFeed: [X-Nav Technical Report: Real-World Long-Horizon Indoor-to-Outdoor Navigation on a Humanoid Robot](<https://devfeed.tech/articles/x-nav-technical-report-real-world-long-horizon-indoor-to-outdoor-navigation-on-a-humanoid-robot-135084.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.17598>)

Author: Jiaxin Huang, Chengxing Lin, Xiaoyu Luo, Sirui Chen, Jingyu Guo, Aolin Li, Kairui Chen, Rui Zhen, Cheng Wen, Kerui Gu, Shunyao Wang, Zhihui Peng, Shaoli Huang

Published: 2026-11-16T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Robot Navigation](<https://devfeed.tech/topics/robot-navigation.md>), [Humanoid Robots](<https://devfeed.tech/topics/humanoid-robots.md>), [Physical AI](<https://devfeed.tech/topics/physical-ai.md>), [Computer vision](<https://devfeed.tech/topics/computer-vision.md>)

Tags: [agentic-workflow](<https://devfeed.tech/tags/agentic-workflow.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-ro](<https://devfeed.tech/tags/cs-ro.md>), [digital-twin](<https://devfeed.tech/tags/digital-twin.md>), [humanoid](<https://devfeed.tech/tags/humanoid.md>), [humanoid-robot](<https://devfeed.tech/tags/humanoid-robot.md>), [lidar](<https://devfeed.tech/tags/lidar.md>), [navigation](<https://devfeed.tech/tags/navigation.md>), [outdoor](<https://devfeed.tech/tags/outdoor.md>), [outdoors](<https://devfeed.tech/tags/outdoors.md>), [pipeline](<https://devfeed.tech/tags/pipeline.md>), [real-world](<https://devfeed.tech/tags/real-world.md>), [rgb](<https://devfeed.tech/tags/rgb.md>), [route](<https://devfeed.tech/tags/route.md>), [site](<https://devfeed.tech/tags/site.md>)

### AI overview

X-Nav demonstrates a language-instructed, roughly 100-metre indoor-to-outdoor mission on the Lingxi X2 humanoid, from an office lobby to a store in a neighboring building. Its workflow, X-Navarch, breaks a brief instruction into subtasks and deploys MELIOR for target navigation and obstacle avoidance. MELIOR uses the robot's built-in RGB camera and has two variants: MELIOR-A predicts actions, while MELIOR-N predicts a navigation goal. The demonstrated mission included a destination change and passage through a door; with X-Navarch, the robot also avoided static and dynamic obstacles. LiDAR and odometry supported target approach and low-level control. Site-specific adaptation used digital-twin data and improved closed-loop success in the twin before real-site deployment. MELIOR also achieved competitive results on R2R-CE and RxR-CE; the excerpt gives no quantitative benchmark results.

### Source excerpt

arXiv:2611.17598v1 Announce Type: cross Abstract: Long-horizon navigation from indoors to outdoors remains an open problem for humanoid robots: one instruction covers rooms, doors and roads, and the camera moves with the gait. Worken, AGIBOT's measure of productive robot work, depends on robots that reach their workplaces autonomously. We present X-Nav, to our knowledge the first system to take a bipedal humanoid through a language-instructed long-horizon mission from indoors to outdoors. X-Nav covers a pipeline from data to deployment. Its data engine collects navigation data in open-source scenes, in digital twins of deployment sites and on real robots. Its navigation model, MELIOR, takes only RGB images from the robot's built-in camera and comes in two versions: MELIOR-A predicts actions, and MELIOR-N predicts a navigation goal. Its agentic workflow, X-Navarch, turns a brief human instruction into sub-tasks and deploys MELIOR on the robot with explicit target navigation and obstacle avoidance. On the Lingxi X2, X-Nav completes a mission of about a hundred metres from an office lobby to a store in a neighbouring building; LiDAR and odometry are used for target approach and low-level control. The mission is completed with both versions of MELIOR: the robot follows a change of destination and walks through a door from the lobby to the road, and with X-Navarch it also passes static and dynamic obstacles on the route. During deployment, MELIOR is adapted with data collected in a digital twin of the site; the adaptation raises its closed-loop success rate in the twin, and the adapted model is then deployed on the robot at the real site. In addition, MELIOR achieves competitive results on the R2R-CE and RxR-CE benchmarks.

## What Is Model Distillation for Computer Vision?

DevFeed: [What Is Model Distillation for Computer Vision?](<https://devfeed.tech/articles/what-is-model-distillation-for-computer-vision-134637.md>)

Original publisher: [Read original article](<https://roboflow.com/blog/model-distillation>)

Author: Timothy M

Published: 2026-11-15T23:35:22Z

Content type: article

Language: en

Sources: [Roboflow](<https://devfeed.tech/sources/roboflow-blog.md>)

Topics: [Model Optimizer](<https://devfeed.tech/topics/model-optimizer.md>), [Computer vision](<https://devfeed.tech/topics/computer-vision.md>), [object-detection](<https://devfeed.tech/topics/object-detection.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [vlm](<https://devfeed.tech/topics/vlm.md>), [Advanced AI workflows](<https://devfeed.tech/topics/advanced-ai-workflows.md>)

Tags: [acceleration](<https://devfeed.tech/tags/acceleration.md>), [annotation](<https://devfeed.tech/tags/annotation.md>), [annotations](<https://devfeed.tech/tags/annotations.md>), [auto-label](<https://devfeed.tech/tags/auto-label.md>), [computer-vision](<https://devfeed.tech/tags/computer-vision.md>), [distillation](<https://devfeed.tech/tags/distillation.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [model-distillation](<https://devfeed.tech/tags/model-distillation.md>), [review](<https://devfeed.tech/tags/review.md>), [student](<https://devfeed.tech/tags/student.md>), [train](<https://devfeed.tech/tags/train.md>), [training](<https://devfeed.tech/tags/training.md>), [workflows](<https://devfeed.tech/tags/workflows.md>)

### AI overview

A large vision model can label training images for a smaller detector that runs locally, but its predictions need human review and the resulting student must be evaluated on the target hardware. The Roboflow workflow described uses either a Gemini 3.1 Pro Workflow or Auto Label to generate face boxes, then trains RF-DETR Nano on the reviewed dataset. These are two labeling paths; the guide says this approach uses supervised training from annotations, not classical soft-target distillation or DeiT's hard-label loss. Compare labeling options using measured usage, current charges, and correction effort, since the text gives no universal cost winner. Check accuracy, capture-to-result latency, memory, and system power on the intended device; compactness alone does not guarantee battery savings. Local inference removes per-frame teacher calls, but fully offline operation has separate Roboflow Inference requirements, including initial authorization and cached model weights.

### Source excerpt

Understand model distillation with Roboflow, where a large teacher model generates labels to help train a smaller student. Follow the process from teacher selection and annotation review to student training and evaluation, using Workflows and Auto Label to prepare data for efficient computer vision.

[Next page](<https://devfeed.tech/topics/computer-vision.md?cursor=WyIyMDI2LTExLTE1VDIzOjM1OjIyKzAwOjAwIiwgImFkMDUyNjQzLTlmMzAtNDhhYS05MGU5LTg1YzMzYjg0MWVlMSJd>)