# Machine Perception

A computing discipline concerned with enabling embodied devices to perceive, comprehend, and reason about their surroundings using sensing and scene-understanding algorithms.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## Backbone Matters: Benchmarking Encoder-Decoder Networks for Semantic Segmentation of Driving Scenes on BDD100K

DevFeed: [Backbone Matters: Benchmarking Encoder-Decoder Networks for Semantic Segmentation of Driving Scenes on BDD100K](<https://devfeed.tech/articles/backbone-matters-benchmarking-encoder-decoder-networks-for-semantic-segmentation-of-driving-scenes-on-bdd100k-145151.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.26191>)

Author: Ehsan Rassekh

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [autonomous vehicles](<https://devfeed.tech/topics/autonomous-vehicles.md>), [Machine Perception](<https://devfeed.tech/topics/machine-perception.md>), [image recognition](<https://devfeed.tech/topics/image-recognition.md>)

Tags: [benchmarking](<https://devfeed.tech/tags/benchmarking.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [decoder](<https://devfeed.tech/tags/decoder.md>), [driving](<https://devfeed.tech/tags/driving.md>), [encoder](<https://devfeed.tech/tags/encoder.md>), [encoder-decoder](<https://devfeed.tech/tags/encoder-decoder.md>), [networks](<https://devfeed.tech/tags/networks.md>), [semantic](<https://devfeed.tech/tags/semantic.md>)

### AI overview

On BDD100K, decoder choice strongly changes how a backbone performs: EfficientNet-B4 scores 62.6% mIoU with FPN but 42.6% with PSPNet. The study trained 52 combinations of four decoders (U-Net, FPN, LinkNet and PSPNet) and 13 backbones across six families using a common protocol. FPN with EfficientNet-B4 was best, reaching 62.6% mIoU, 89.8% accuracy and 0.092 loss; only two FPN-EfficientNet combinations exceeded the cited BDD100K benchmark of 56.9% mIoU. The best-to-worst backbone gap varied by decoder, from 12.0 points for U-Net to 42.5 for FPN. These are reported results for this dataset and protocol; the supplied text does not provide further methodological details or broader validation.

### Source excerpt

arXiv:2611.26191v1 Announce Type: cross Abstract: Semantic segmentation gives an autonomous vehicle a dense, pixel-level description of its surroundings and is a core part of scene understanding. Most segmentation networks consist of an encoder (the backbone) and a decoder, yet the backbone is often chosen without a systematic comparison. This work studies this choice on the BDD100K driving dataset. Four widely used decoders (U-Net, FPN, LinkNet and PSPNet) are combined with 13 backbones from six families (ResNet, VGG, DenseNet, Inception, MobileNet and EfficientNet), and all 52 models are trained with one common protocol. The best model, an FPN decoder with an EfficientNet-B4 backbone, reaches 62.6% mean intersection over union (mIoU), 89.8% accuracy and a loss of 0.092. Only two of the 52 models, both FPN with EfficientNet, exceed the 56.9% mIoU reported with the BDD100K benchmark. The effect of the backbone depends strongly on the decoder: the same EfficientNet-B4 gives 42.6% mIoU with PSPNet but 62.6% with FPN, and the gap between the best and the worst backbone ranges from 12.0 points (U-Net) to 42.5 points (FPN). The results and the scope of this study are discussed.

## LoongForge-Embodied: A Unified Framework for Training Heterogeneous Embodied Models

DevFeed: [LoongForge-Embodied: A Unified Framework for Training Heterogeneous Embodied Models](<https://devfeed.tech/articles/loongforge-embodied-a-unified-framework-for-training-heterogeneous-embodied-models-145515.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.25460>)

Author: LoongForge Team

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Advanced AI workflows](<https://devfeed.tech/topics/advanced-ai-workflows.md>), [Machine Perception](<https://devfeed.tech/topics/machine-perception.md>), [Distributed Systems & Parallel Computing](<https://devfeed.tech/topics/distributed-systems-parallel-computing.md>)

Tags: [acceleration](<https://devfeed.tech/tags/acceleration.md>), [architectures](<https://devfeed.tech/tags/architectures.md>), [concurrency](<https://devfeed.tech/tags/concurrency.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [data](<https://devfeed.tech/tags/data.md>), [data-infrastructure](<https://devfeed.tech/tags/data-infrastructure.md>), [ddp](<https://devfeed.tech/tags/ddp.md>), [distributed](<https://devfeed.tech/tags/distributed.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [execution](<https://devfeed.tech/tags/execution.md>), [fp8](<https://devfeed.tech/tags/fp8.md>), [framework](<https://devfeed.tech/tags/framework.md>), [fsdp](<https://devfeed.tech/tags/fsdp.md>), [hardware](<https://devfeed.tech/tags/hardware.md>), [i-o](<https://devfeed.tech/tags/i-o.md>), [infrastructure](<https://devfeed.tech/tags/infrastructure.md>), [layer](<https://devfeed.tech/tags/layer.md>), [memory](<https://devfeed.tech/tags/memory.md>), [native](<https://devfeed.tech/tags/native.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [training](<https://devfeed.tech/tags/training.md>)

### AI overview

LoongForge-Embodied uses a shared abstraction layer to make training infrastructure reusable across heterogeneous VLA and WAM architectures without constraining model-specific execution needs. The native-PyTorch framework combines shared and model- or hardware-specific optimizations for computation, memory, data I/O, execution, and distributed communication, including FP8 communication for DDP and FSDP. It also includes infrastructure for synthesizing robot data from egocentric data and concurrent simulator evaluation. Across representative models, the paper reports 1.47x-4.38x higher end-to-end training throughput than corresponding open-source baselines and 4.1x-23.2x faster evaluation under the same protocols.

### Source excerpt

arXiv:2611.25460v1 Announce Type: new Abstract: Training Vision-Language-Action (VLA) models and World-Action Models (WAMs) is computationally expensive, while their heterogeneous architectures and fragmented training stacks make system optimizations difficult to reuse across models. To address these challenges, we present LoongForge-Embodied, a native-PyTorch framework that unifies and accelerates the training of heterogeneous embodied models. At its core, LoongForge-Embodied introduces a common abstraction layer that decouples model-specific implementations from shared training infrastructure, enabling cross-model reuse without constraining individual architectures or execution requirements. Buildingon this unified infrastructure, the framework combines reusable acceleration mechanisms with model-specific and hardware-aware optimization strategies across computation, execution, memory, data I/O, and distributed communication, including FP8-based communication optimizations for DDP and FSDP. Beyond training acceleration, LoongForge-Embodied integrates a unified data infrastructure with a pipeline for synthesizing robot data from egocentric data, and an efficient evaluation framework that supports concurrency and optimized computation in simulator. Experiments across representative VLA and WAM models demonstrate 1.47x-4.38x end-to-end training throughput improvements over their corresponding open-source baselines, as well as 4.1x-23.2x faster evaluation under the same protocols.

## Towards Embodied Air-Ground Cooperative Object Search: Benchmark, Dataset and Agentic Method

DevFeed: [Towards Embodied Air-Ground Cooperative Object Search: Benchmark, Dataset and Agentic Method](<https://devfeed.tech/articles/towards-embodied-air-ground-cooperative-object-search-benchmark-dataset-and-agentic-method-145369.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2609.08402>)

Author: Boao Yu, Zimo Chen, Junreng Rao, Yue Hu, Zhengqiu Zhu, Yong Zhao, Rusheng Ju

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Machine Perception](<https://devfeed.tech/topics/machine-perception.md>), [Multi-step AI processes](<https://devfeed.tech/topics/multi-step-ai-processes.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Image](<https://devfeed.tech/topics/image.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [agentic](<https://devfeed.tech/tags/agentic.md>), [air](<https://devfeed.tech/tags/air.md>), [approach](<https://devfeed.tech/tags/approach.md>), [automatic](<https://devfeed.tech/tags/automatic.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [benchmark-dataset](<https://devfeed.tech/tags/benchmark-dataset.md>), [computer-science](<https://devfeed.tech/tags/computer-science.md>), [computer-vision](<https://devfeed.tech/tags/computer-vision.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [dataset](<https://devfeed.tech/tags/dataset.md>), [decision-making](<https://devfeed.tech/tags/decision-making.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [first](<https://devfeed.tech/tags/first.md>), [gemini](<https://devfeed.tech/tags/gemini.md>), [gemini-3-6-flash](<https://devfeed.tech/tags/gemini-3-6-flash.md>), [object](<https://devfeed.tech/tags/object.md>), [search](<https://devfeed.tech/tags/search.md>), [vision-language-models-vlms](<https://devfeed.tech/tags/vision-language-models-vlms.md>), [vlms](<https://devfeed.tech/tags/vlms.md>)

### AI overview

The paper introduces AGOS, an urban search task where UAVs find a target vehicle from multi-view references and UGVs verify it at ground level. AGOS-Bench evaluates whether general-purpose vision-language models can coordinate those roles; its companion AGOS-Dataset contains 7.7k automatically constructed episodes across three difficulty levels. The proposed training-free, tool-augmented AGOS-Agent uses a search-handoff-verify protocol, leaving scene understanding and decision-making to the VLMs. Across nine evaluated models, it improves overall success rate for eight and reduces decision steps for all nine. On the hard split, Gemini-3.6-Flash's success rate rises from 8.6% to 55.7%, and its SPL from 7.6% to 44.0%.

### Source excerpt

arXiv:2609.08402v2 Announce Type: replace-cross Abstract: Air-Ground Object Search (AGOS) in urban environments is a challenging embodied task, which requires an Unmanned Aerial Vehicle (UAV) and an Unmanned Ground Vehicle (UGV) to jointly search for and verify a specified target vehicle from multi-view visual references. To study this underexplored problem, we introduce AGOS-Bench, the first dedicated benchmark for evaluating whether general-purpose Vision-Language Models (VLMs) can integrate aerial discoveries and ground-level verification through UAV-UGV cooperation. We further provide AGOS-Dataset as the companion resource of exemplary trajectories constructed by an automatic pipeline. It consists of 7.7k episodes for searching objects of diverse categories and attributes, spanning three difficulty levels. To address the AGOS task, we propose AGOS-Agent, a training-free and tool-augmented approach. The agentic method relieves VLMs from complex and dynamic coordination via a deliberate search-handoff-verify cooperation protocol, only demanding VLMs for scene understanding and decision-making. Extensive experiments on nine VLMs show that AGOS-Agent improves overall success rate for eight of the nine evaluated backbones while reducing decision steps for all nine. On the hard split, the SR and SPL of Gemini-3.6-Flash increase from 8.6% to 55.7% and from 7.6% to 44.0%, respectively.

## AeroBelief: Budgeted First-Action Verification for Evidence-Guided Aerial Sensing

DevFeed: [AeroBelief: Budgeted First-Action Verification for Evidence-Guided Aerial Sensing](<https://devfeed.tech/articles/aerobelief-budgeted-first-action-verification-for-evidence-guided-aerial-sensing-145539.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.25568>)

Author: Tianren Li, Dongbo Li, Hongkai Chen, Jie Liu, Guoliang Xing

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Machine Perception](<https://devfeed.tech/topics/machine-perception.md>), [Image processing](<https://devfeed.tech/topics/image-processing.md>), [Image](<https://devfeed.tech/topics/image.md>)

Tags: [budgeting](<https://devfeed.tech/tags/budgeting.md>), [complete](<https://devfeed.tech/tags/complete.md>), [context](<https://devfeed.tech/tags/context.md>), [controller](<https://devfeed.tech/tags/controller.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [dji](<https://devfeed.tech/tags/dji.md>), [feedback](<https://devfeed.tech/tags/feedback.md>), [hosted](<https://devfeed.tech/tags/hosted.md>), [image](<https://devfeed.tech/tags/image.md>), [image-processing](<https://devfeed.tech/tags/image-processing.md>), [imagery](<https://devfeed.tech/tags/imagery.md>), [queries](<https://devfeed.tech/tags/queries.md>), [reference](<https://devfeed.tech/tags/reference.md>), [sensing](<https://devfeed.tech/tags/sensing.md>), [target](<https://devfeed.tech/tags/target.md>), [verification](<https://devfeed.tech/tags/verification.md>), [world-model](<https://devfeed.tech/tags/world-model.md>)

### AI overview

AeroBelief verifies candidate aerial views by their next executable action: an inexpensive proposer scans grouped views, then a structured world model is queried selectively. Calibrated bounds and a six-query controller connect action selection to belief updates, acceptance checks, and scheduled sensing. In 960 executions across 40 worlds, it completed 75.42% of tasks, versus 73.33% with exhaustive world-model scoring and 65.83% with line-of-sight geometry, while using 91.81% fewer reference-candidate evaluations than exhaustive scoring. In fixed-reference replay, proposer-scaled bounds improved first-action certificate availability from 36.25% to 54.69% and cut mean queries per state by 13.54% versus unit-scale anchored bounds. A separate ground-hosted pipeline applied context-guided admission and feedback to 2,685 UAV images from seven routes: it retained all eight independently supported target confirmations while processing 76.05% fewer images and using 70.02% less active processing service than uncapped all-frame processing. These findings are reported for the described simulations and recorded imagery.

### Source excerpt

arXiv:2611.25568v1 Announce Type: new Abstract: Aerial sensing must combine uncertain overhead context with current observations while budgeting prediction, movement and image processing. AeroBelief uses the next executable action as the unit of verification against reference scores. It groups candidate views by first action, scans them with an inexpensive proposer, and selectively queries a structured world model (WM). Calibrated group bounds support strict commitment; a six-query controller connects selective prediction with belief updates, acceptance checks and scheduled sensing. Across four layout families, 40 worlds and 960 closed-loop executions, the AeroBelief controller completes 75.42% of tasks, compared with 73.33% for exhaustive WM scoring and 65.83% for line-of-sight geometry, using 91.81% fewer reference-candidate evaluations than exhaustive scoring. In fixed-reference replay with a six-query cap, proposer-scaled anchored bounds raised first-action certificate availability from 36.25% to 54.69% and reduced mean queries per state by 13.54% relative to unit-scale anchored bounds. A complete ground-hosted pipeline processes 2,685 self-collected 20-megapixel unmanned aerial vehicle (UAV) images from seven DJI Mavic 3M routes. Context-guided admission and feedback retain all eight independently supported target confirmations with 76.05% fewer processed images and 70.02% less accumulated active processing service than uncapped all-frame processing; feedback saves another 5.17% of images relative to static context. The studies connect budgeted action selection with evidence-driven processing of recorded flight imagery.

## RECLAIM: Robot-Vision-Informed Adversarial Patch Defense via Embodied Attention Feedback

DevFeed: [RECLAIM: Robot-Vision-Informed Adversarial Patch Defense via Embodied Attention Feedback](<https://devfeed.tech/articles/reclaim-robot-vision-informed-adversarial-patch-defense-via-embodied-attention-feedback-143156.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.24494>)

Author: Jiayou Zhu, Peng Guo, Litian Zhang, Xinze Shi, Yuting Zhang, Zelong Yu, Yize Tian, Jirong Liu, Zifeng Kang

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Cryptography and Security](<https://devfeed.tech/sources/arxiv-cryptography-and-security.md>)

Topics: [Adversarial attacks](<https://devfeed.tech/topics/adversarial-attacks.md>), [Robot Manipulation](<https://devfeed.tech/topics/robot-manipulation.md>), [Machine Perception](<https://devfeed.tech/topics/machine-perception.md>)

Tags: [actions](<https://devfeed.tech/tags/actions.md>), [adversarial](<https://devfeed.tech/tags/adversarial.md>), [agent](<https://devfeed.tech/tags/agent.md>), [attack](<https://devfeed.tech/tags/attack.md>), [attention](<https://devfeed.tech/tags/attention.md>), [changes](<https://devfeed.tech/tags/changes.md>), [control](<https://devfeed.tech/tags/control.md>), [cs-cr](<https://devfeed.tech/tags/cs-cr.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-ro](<https://devfeed.tech/tags/cs-ro.md>), [decision](<https://devfeed.tech/tags/decision.md>), [defense](<https://devfeed.tech/tags/defense.md>), [edits](<https://devfeed.tech/tags/edits.md>), [effective](<https://devfeed.tech/tags/effective.md>), [feedback](<https://devfeed.tech/tags/feedback.md>), [localization](<https://devfeed.tech/tags/localization.md>), [manipulation](<https://devfeed.tech/tags/manipulation.md>), [models](<https://devfeed.tech/tags/models.md>), [multiple](<https://devfeed.tech/tags/multiple.md>), [patch](<https://devfeed.tech/tags/patch.md>), [patches](<https://devfeed.tech/tags/patches.md>), [robot](<https://devfeed.tech/tags/robot.md>), [vision](<https://devfeed.tech/tags/vision.md>)

### AI overview

RECLAIM is a training-free, inference-time defense that uses a frozen vision-language-action policy's attention response to local image edits to guide patch localization and purification during robot manipulation. It combines that feedback with robot geometry, then revalidates treatments as actions change the observed scene. In evaluation against patches from four attack methods across multiple task suites, it achieved at least one successful rollout on 95% of evaluated tasks and improved LIBERO-Object rollout success from 0% without defense to 70.0%. Mean decision time was 548.9 ms per control step. The abstract reports these results under frozen VLA control; it does not detail the specific local treatments.

### Source excerpt

arXiv:2611.24494v1 Announce Type: cross Abstract: Vision-language-action policies control embodied agents through a perception-action loop grounded in physical interaction, where adversarial patches can corrupt action predictions and thereby alter subsequent observations. To mitigate these threats, many existing defenses rely on policy adaptation or auxiliary models, but require additional training or inference. We introduce RECLAIM, a training-free, inference-time defense that uses a frozen VLA's own attention response to local visual interventions to guide purification during closed-loop embodied manipulation. Our key insight is that attention changes induced by local edits provide feedback for choosing where and how to purify as the robot interacts with its environment. RECLAIM combines this feedback with robot geometry to refine patch localization and select local treatments, then revalidates them as the agent's embodied actions change the observed scene. We evaluate RECLAIM against patches generated by four adversarial attack methods across multiple manipulation task suites under frozen VLA control. Under adversarial patch attacks, RECLAIM succeeds in at least one rollout on 95% of the evaluated tasks and raises rollout success on LIBERO-Object from 0% without defense to 70.0%. Its mean decision time is only 548.9 ms per control step. These results highlight policy-guided visual purification as an effective approach to defending embodied agents against adversarial patches and recovering closed-loop task execution.

## ReflectScene:MLLM-Guided Closed-Loop Diffusion for 3D Indoor Scene Generation

DevFeed: [ReflectScene:MLLM-Guided Closed-Loop Diffusion for 3D Indoor Scene Generation](<https://devfeed.tech/articles/reflectscene-mllm-guided-closed-loop-diffusion-for-3d-indoor-scene-generation-142710.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.23845>)

Author: Jiawei Shen, Jie Feng, Junpeng Zhang, Guanbin Li, Wanyi Chen, Weisheng Dong, Guangming Shi

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Machine Perception](<https://devfeed.tech/topics/machine-perception.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [ai-image-generator](<https://devfeed.tech/topics/ai-image-generator.md>)

Tags: [3d](<https://devfeed.tech/tags/3d.md>), [alignment](<https://devfeed.tech/tags/alignment.md>), [consistency](<https://devfeed.tech/tags/consistency.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [denoising](<https://devfeed.tech/tags/denoising.md>), [diffusion](<https://devfeed.tech/tags/diffusion.md>), [diversity](<https://devfeed.tech/tags/diversity.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [feedback](<https://devfeed.tech/tags/feedback.md>), [functional](<https://devfeed.tech/tags/functional.md>), [generation](<https://devfeed.tech/tags/generation.md>), [large-language-model](<https://devfeed.tech/tags/large-language-model.md>), [layout](<https://devfeed.tech/tags/layout.md>), [layouts](<https://devfeed.tech/tags/layouts.md>), [quality](<https://devfeed.tech/tags/quality.md>), [reward](<https://devfeed.tech/tags/reward.md>), [scene-generation](<https://devfeed.tech/tags/scene-generation.md>), [semantic](<https://devfeed.tech/tags/semantic.md>), [semantics](<https://devfeed.tech/tags/semantics.md>)

### AI overview

ReflectScene uses a closed-loop diffusion process to make object-level corrections to generated 3D indoor layouts while preserving non-target arrangements during denoising. A frozen multimodal large language model interprets structured layouts alongside multi-view visual evidence, and its assessments of candidate corrections are distilled into a lightweight reward predictor to reduce correction-decision computation. The selected corrections are applied through transient conditioning and persistent clean-estimate projection. On 3D-FRONT, the authors report improved functional consistency and semantic alignment with competitive generation quality and diversity.

### Source excerpt

arXiv:2611.23845v1 Announce Type: cross Abstract: Text-conditioned 3D indoor scene generation requires layouts satisfying geometric, functional, and semantic constraints. However, existing diffusion methods rely primarily on static textual conditions, limiting their ability to assess evolving scenes and translate semantic discrepancies into executable object-level corrections. MLLM-based post-hoc refinement provides semantic feedback but remains decoupled from denoising, potentially disrupting scene coherence.We propose ReflectScene, an MLLM-guided closed-loop diffusion framework. First, we introduce Hierarchical Prior-Grounded Scene Understanding that aligns structured layouts with multi-view visual evidence, to interpret object-specific context and scene global semantics via a frozen multi-modal large language model (MLLM). Second, we propose MLLM-Guided Semantic Reward Evaluation to distill MLLM's diagnoses and assessments on possible candidate corrections into a lightweight Reward Predictor, enabling correction decision making with reduced computation burden. Finally, Decision-Conditioned Trajectory-Consistent Diffusion is proposed to integrate selected corrections through transient conditioning and persistent clean-estimate projection, correcting object layout in the generated scene while preserving non-target arrangements throughout denoising.Experiments on 3D-FRONT demonstrate improved functional consistency and semantic alignment while maintaining competitive generation quality and diversity.

## Meet the Anzellans: Inside Imagineering R&D's newest robotic characters

DevFeed: [Meet the Anzellans: Inside Imagineering R&D's newest robotic characters](<https://devfeed.tech/articles/meet-the-anzellans-inside-imagineering-r-d-s-newest-robotic-characters-143691.md>)

Original publisher: [Read original article](<https://aws.amazon.com/blogs/media/meet-the-anzellans-inside-imagineering-rds-newest-robotic-characters/>)

Author: Ari Entin

Published: 2026-11-18T20:03:05Z

Content type: news

Language: en

Sources: [AWS for M&E Blog](<https://devfeed.tech/sources/aws-for-m-e-blog.md>)

Topics: [Physical AI](<https://devfeed.tech/topics/physical-ai.md>), [Machine Perception](<https://devfeed.tech/topics/machine-perception.md>), [Humanoid Robots](<https://devfeed.tech/topics/humanoid-robots.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [animations](<https://devfeed.tech/tags/animations.md>), [animatronics](<https://devfeed.tech/tags/animatronics.md>), [audio](<https://devfeed.tech/tags/audio.md>), [aws](<https://devfeed.tech/tags/aws.md>), [aws-re-invent](<https://devfeed.tech/tags/aws-re-invent.md>), [cloud](<https://devfeed.tech/tags/cloud.md>), [cloud-infrastructure](<https://devfeed.tech/tags/cloud-infrastructure.md>), [collaboration](<https://devfeed.tech/tags/collaboration.md>), [customer-solutions](<https://devfeed.tech/tags/customer-solutions.md>), [decision-making](<https://devfeed.tech/tags/decision-making.md>), [decision-models](<https://devfeed.tech/tags/decision-models.md>), [disney](<https://devfeed.tech/tags/disney.md>), [disney-plus](<https://devfeed.tech/tags/disney-plus.md>), [disneyland](<https://devfeed.tech/tags/disneyland.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [english](<https://devfeed.tech/tags/english.md>), [intermediate-200](<https://devfeed.tech/tags/intermediate-200.md>), [prototype](<https://devfeed.tech/tags/prototype.md>), [real-time](<https://devfeed.tech/tags/real-time.md>), [robotic](<https://devfeed.tech/tags/robotic.md>), [walt-disney-imagineering](<https://devfeed.tech/tags/walt-disney-imagineering.md>)

### AI overview

The Anzellans are Audio-Animatronics prototypes that respond to guests and one another in real time instead of repeating a fixed performance. AWS powers their real-time intelligence: fast-decision models support quick reactions, while larger models weigh context. The system selects prerecorded audio and pairs it with dynamic animations, with components optimized for latency. Imagineers created the dialogue, gestures and personalities, keeping the characters within the designed story. The excerpt describes a prototype and an AWS re:Invent showcase; it does not establish a broader public deployment.

### Source excerpt

Meet the Anzellans -- Vilo Vele, Kado Bre'a, and Jibu Bre'a -- the newest Audio-Animatronics character prototype from Walt Disney Imagineering R&D. Built with AWS, these autonomous characters don't run a fixed loop. They perceive guests in real time, decide how to respond, and act on their own, all within the story Imagineers designed.

## NexSight: Verifiable Tasks for Fine-Grained Agentic Long-Video Understanding

DevFeed: [NexSight: Verifiable Tasks for Fine-Grained Agentic Long-Video Understanding](<https://devfeed.tech/articles/nexsight-verifiable-tasks-for-fine-grained-agentic-long-video-understanding-140080.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.22790>)

Author: Tianci Xie, Li Chen, Pinxue Guo, Rui Zheng

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [AI Foundation Models](<https://devfeed.tech/topics/ai-foundation-models.md>), [Machine Perception](<https://devfeed.tech/topics/machine-perception.md>), [post-training](<https://devfeed.tech/topics/post-training.md>)

Tags: [agentic](<https://devfeed.tech/tags/agentic.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [exploration](<https://devfeed.tech/tags/exploration.md>), [fine](<https://devfeed.tech/tags/fine.md>), [harness](<https://devfeed.tech/tags/harness.md>), [inspection](<https://devfeed.tech/tags/inspection.md>), [integration](<https://devfeed.tech/tags/integration.md>), [interactive](<https://devfeed.tech/tags/interactive.md>), [pipeline](<https://devfeed.tech/tags/pipeline.md>), [post-training](<https://devfeed.tech/tags/post-training.md>), [raw](<https://devfeed.tech/tags/raw.md>), [requires](<https://devfeed.tech/tags/requires.md>), [spatial](<https://devfeed.tech/tags/spatial.md>), [supervision](<https://devfeed.tech/tags/supervision.md>), [tasks](<https://devfeed.tech/tags/tasks.md>), [this](<https://devfeed.tech/tags/this.md>), [times](<https://devfeed.tech/tags/times.md>), [train](<https://devfeed.tech/tags/train.md>), [training](<https://devfeed.tech/tags/training.md>), [video](<https://devfeed.tech/tags/video.md>)

### AI overview

NexSight addresses fine-grained long-video questions by building tasks whose answers must be reconstructible from densely reacquired video evidence and survive counterexample checks. Its SightForge pipeline creates SightLine, a set of 310 evidence-traceable tasks from 108 long videos, evaluated with SightLoop, which supports multiscale temporal search, local inspection, evidence integration, and revisiting while logging observations. Solver trajectories on disjoint videos provide step-level supervision for observation choices. The strongest model scores 54.52% on SightLine, rising to 79.35% when given annotated evidence. With under 6K examples, Sight-35B improves interactive accuracy from 21.29% to 42.26%, exceeds its family's 397B model, starts using explicit spatial inspection, and gains over 8 points on two external benchmarks. These results indicate that evidence access and search behavior remain substantial challenges.

### Source excerpt

arXiv:2611.22790v1 Announce Type: cross Abstract: Fine-grained questions about long videos ask how many times an action occurs, in what order events unfold, or exactly when a state changes. Answering them can require finding brief moments scattered across hours of footage. Agentic models that decide when and where to look promise to find these moments, yet this ability is hard both to measure and to learn. Sparse supporting evidence may establish that an action occurred without establishing its total count; verifying timeline-wide claims requires checking for missed occurrences and double counting. Answer-only supervision, in turn, does not explicitly supervise the search for evidence. We present NexSight, which builds verifiable tasks to both evaluate and train agentic long-video understanding. Its agentic pipeline, SightForge, accepts a task only if a verifier blind to the draft answer can reconstruct it from evidence densely reacquired from the raw video and counterexample checks fail to refute it. This yields SightLine, 310 evidence-traceable tasks from 108 long videos, evaluated through SightLoop, a shared multiscale harness supporting temporal exploration, local inspection, evidence integration, and revisiting while recording returned observations. On disjoint videos, solver trajectories on verified tasks become step-level supervision for observation decisions. The strongest model reaches only 54.52% on SightLine, yet 79.35% when given the annotated evidence. With under 6K examples, Sight-35B nearly doubles its base model's interactive accuracy (21.29% to 42.26%), outperforms the 397B model of its family, begins using explicit spatial inspection after post-training, which its base model never uses, and gains over 8 points on two external long-video benchmarks.

## Radar Panoptic Transformer: Semantic-Guided Mask Queries for Dynamic Scene Understanding

DevFeed: [Radar Panoptic Transformer: Semantic-Guided Mask Queries for Dynamic Scene Understanding](<https://devfeed.tech/articles/radar-panoptic-transformer-semantic-guided-mask-queries-for-dynamic-scene-understanding-139008.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.20203>)

Author: Mayank Mayank, Bharanidhar Duraisamy, Rohit Mohan, Abhinav Valada

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Machine Perception](<https://devfeed.tech/topics/machine-perception.md>), [Time-sequences](<https://devfeed.tech/topics/time-sequences.md>), [Transformer](<https://devfeed.tech/topics/transformer.md>)

Tags: [attention](<https://devfeed.tech/tags/attention.md>), [computer-vision](<https://devfeed.tech/tags/computer-vision.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [doppler](<https://devfeed.tech/tags/doppler.md>), [mask](<https://devfeed.tech/tags/mask.md>), [network](<https://devfeed.tech/tags/network.md>), [queries](<https://devfeed.tech/tags/queries.md>), [radar](<https://devfeed.tech/tags/radar.md>), [rcs](<https://devfeed.tech/tags/rcs.md>), [real-time](<https://devfeed.tech/tags/real-time.md>), [semantic](<https://devfeed.tech/tags/semantic.md>), [temporal](<https://devfeed.tech/tags/temporal.md>), [transformer](<https://devfeed.tech/tags/transformer.md>)

### AI overview

Radar Panoptic Transformer (RPT) addresses sparse radar point clouds by combining pose-aligned temporal features and per-point neighborhoods adapted to range, azimuth, and radar cross section (RCS). Point-wise semantic predictions seed foreground queries and guide masked cross-attention to produce instance masks without geometric clustering; an auxiliary loss encourages predicted and annotated instances to have consistent RCS statistics. The abstract reports 71.6% mIoU and 82.2% PQ on RadarScenes, and says RPT outperforms evaluated baselines on MAN TruckScenes in a full-surround setting with sensing ranges up to 230 m. RadarScenes inference runs at 31 Hz, above the sensor frame rate.

### Source excerpt

arXiv:2611.20203v1 Announce Type: cross Abstract: Automotive radar provides long-range sensing and direct Doppler velocity measurements, but its sparse and non-uniformly sampled point clouds pose significant challenges for panoptic segmentation. Dynamic objects often yield only a few detections amid a dominant static background, providing limiting the geometric cues for separating object instances. We propose \emph{Radar Panoptic Transformer (RPT)}, an end-to-end point-based network that incorporates radar-adaptive feature extraction with semantic-guided mask prediction. Its backbone aggregates pose-aligned temporal context through grouped vector attention and learns per-point neighborhood radii conditioned on range, azimuth, and radar cross section (RCS). Point-wise semantic predictions initialize queries at likely foreground detections and guide masked cross-attention through query-point class agreement, producing instance masks without geometric clustering. An auxiliary RCS-consistency loss matches instance-level statistics between predicted and annotated instances. RPT achieves state-of-the-art performance on RadarScenes with $71.6\%$ mIoU and $82.2\%$ PQ. We further evaluate RPT on MAN TruckScenes, where it outperforms the evaluated baselines in a long-range, full-surround radar setting with sensing ranges up to $230\,\mathrm{m}$. RPT achieves real-time end-to-end inference at $31$\,Hz on RadarScenes, exceeding the sensor frame rate.

## Is It Still There? Place-Registered Verification Makes VLMs Reliable Scene-Change Detectors

DevFeed: [Is It Still There? Place-Registered Verification Makes VLMs Reliable Scene-Change Detectors](<https://devfeed.tech/articles/is-it-still-there-place-registered-verification-makes-vlms-reliable-scene-change-detectors-137335.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.18734>)

Author: Jinjie Xie, Baihua Li, Qinggang Meng

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Machine Perception](<https://devfeed.tech/topics/machine-perception.md>), [AI Foundation Models](<https://devfeed.tech/topics/ai-foundation-models.md>)

Tags: [3d](<https://devfeed.tech/tags/3d.md>), [change-detection](<https://devfeed.tech/tags/change-detection.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [kitchen](<https://devfeed.tech/tags/kitchen.md>), [robot](<https://devfeed.tech/tags/robot.md>), [verification](<https://devfeed.tech/tags/verification.md>), [vlms](<https://devfeed.tech/tags/vlms.md>)

### AI overview

STILL improves robot scene-change detection by checking whether each hypothesized object remains at its registered place across two captures. It uses 3D geometry to register that place in the other capture, then has one VLM assess scene and local-place views, with training supervision in the same form. Under the official SceneDiff protocol, obj/sc AP rises from 24.0 to 37.8; on the egocentric kitchen subset, it rises from 11.6 to 20.6 without kitchen training data. The abstract also reports an F1 gain of more than 21 points over a VLM fine-tuned on the same annotations, at about 11% added runtime.

### Source excerpt

arXiv:2611.18734v1 Announce Type: cross Abstract: A robot that returns to a space must tell which objects were added, removed or moved since its last visit, while its trajectory, viewpoints and lighting differ between the two visits. We find that multi-image vision-language models (VLMs) fail at this task mainly because they cannot find the place where an object was. A VLM recognises what each frame shows but cannot check whether an object is still at its place in the other capture. It therefore reports objects that only one capture shows and misses most removed and moved objects. We propose STILL, which reformulates change detection as object verification at a registered place. 3D geometry registers the place of each change hypothesis in the other capture, and one VLM answers, from a scene view and a local place view, whether the object is still at that place, trained with supervision of the same form. Under the official SceneDiff protocol, STILL raises the main metric obj/sc AP from 24.0 to 37.8, and from 11.6 to 20.6 on the egocentric kitchen subset without any kitchen training data. It improves F1 by more than 21 points over a VLM fine-tuned on the same annotations and adds only about 11% to the runtime.

## Different Timescales, Different Signals: Multimodal Landmark-Visibility Prediction in a Long-Term Webcam Network

DevFeed: [Different Timescales, Different Signals: Multimodal Landmark-Visibility Prediction in a Long-Term Webcam Network](<https://devfeed.tech/articles/different-timescales-different-signals-multimodal-landmark-visibility-prediction-in-a-long-term-webcam-network-138780.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.18707>)

Author: Bryceton Bible, Hairong Qi, Masahiro Toyoura

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Machine Perception](<https://devfeed.tech/topics/machine-perception.md>), [Image](<https://devfeed.tech/topics/image.md>)

Tags: [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [forecast](<https://devfeed.tech/tags/forecast.md>), [forecasting](<https://devfeed.tech/tags/forecasting.md>), [multimodal](<https://devfeed.tech/tags/multimodal.md>), [network](<https://devfeed.tech/tags/network.md>), [prediction](<https://devfeed.tech/tags/prediction.md>), [visibility](<https://devfeed.tech/tags/visibility.md>), [webcam](<https://devfeed.tech/tags/webcam.md>)

### AI overview

For Mount Fuji visibility, the most useful prediction signal depends on the lead time: seasonal climatology helps with long-range guidance, webcam imagery is strongest for same-day observation (AUC 0.95), and the numerical forecast valid on the target day carries most day-specific information one to three days ahead. In 6,575 camera-days, adding the current visual summary provided no measurable one-day gain once that forecast was available. Evaluation must account for cameras sharing regional weather: permissive data splits overstated multi-day performance, while date-grouped and forward-in-time evaluation reduced the apparent contribution of current scenes and weather. The authors identify cloud geometry and motion, discarded by the current visual representation, as a remaining opportunity at shorter horizons.

### Source excerpt

arXiv:2611.18707v1 Announce Type: cross Abstract: Scenic visibility is not a single forecasting problem: planning a trip months ahead, deciding whether to go outside today, and judging whether tomorrow will be clear rely on different information. Using a year-scale multimodal archive of Mount Fuji webcams and meteorological observations, we identify a clear hierarchy across timescales. Seasonal climatology provides useful long-range guidance, webcam imagery is the strongest same-day observation (AUC 0.95), and the numerical forecast valid on the target day carries most of the day-specific signal one to three days ahead. Revealing this hierarchy requires evaluation that respects the dependence among cameras observing the same regional weather state: permissive splits substantially overstate multi-day performance, while date-grouped and forward-in-time evaluation reduce the apparent contribution of the current scene and current weather. On 6,575 camera-days, the current visual summary provides no measurable one-day gain once the target-day forecast is available. The remaining opportunity lies at shorter horizons, where the present visual representation discards cloud geometry and motion. These results recast scenic visibility prediction as a multi-timescale sensing problem rather than a single multimodal forecasting task.

## UNITE: Consolidating Embodied VLM Specialists via Shared-Anchor One-Bit Factorization

DevFeed: [UNITE: Consolidating Embodied VLM Specialists via Shared-Anchor One-Bit Factorization](<https://devfeed.tech/articles/unite-consolidating-embodied-vlm-specialists-via-shared-anchor-one-bit-factorization-139093.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.20767>)

Author: Tao Tao, Anke Tang, Enneng Yang, Zhi Hou, Xiaogang Wang, Dacheng Tao

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Machine Perception](<https://devfeed.tech/topics/machine-perception.md>), [Mixture of Experts (MoE)](<https://devfeed.tech/topics/mixture-of-experts-moe.md>), [LLM evaluation / benchmarking](<https://devfeed.tech/topics/llm-evaluation-benchmarking.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [peft](<https://devfeed.tech/topics/peft.md>)

Tags: [3d](<https://devfeed.tech/tags/3d.md>), [3d-scene-understanding](<https://devfeed.tech/tags/3d-scene-understanding.md>), [anchor](<https://devfeed.tech/tags/anchor.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [driving](<https://devfeed.tech/tags/driving.md>), [experts](<https://devfeed.tech/tags/experts.md>), [floating-point](<https://devfeed.tech/tags/floating-point.md>), [mixture-of-experts](<https://devfeed.tech/tags/mixture-of-experts.md>), [model](<https://devfeed.tech/tags/model.md>), [moe](<https://devfeed.tech/tags/moe.md>), [navigation](<https://devfeed.tech/tags/navigation.md>), [router](<https://devfeed.tech/tags/router.md>), [spatial](<https://devfeed.tech/tags/spatial.md>), [spatial-reasoning](<https://devfeed.tech/tags/spatial-reasoning.md>), [specialists](<https://devfeed.tech/tags/specialists.md>), [storage](<https://devfeed.tech/tags/storage.md>), [token](<https://devfeed.tech/tags/token.md>), [unite](<https://devfeed.tech/tags/unite.md>), [vlm](<https://devfeed.tech/tags/vlm.md>)

### AI overview

UNITE combines four embodied vision-language specialists in a shared-backbone mixture of experts, retaining each specialist's feed-forward block as a routable expert. It compresses the expert bank with a shared floating-point anchor and expert-specific row-scaled binary deviations, optimized from source weights without calibration data. Router adaptation freezes the backbone and experts and updates fewer than 0.6 million parameters. The resulting Qwen3-VL-8B model occupies 20.3 GB--1.16 times one dense specialist and 59.6% less than the uncompressed MoE. Across 22 benchmarks and R2R Val-Unseen, it matches six static merging methods in mean retention of best-specialist performance, leads them on Scan2Cap, ScanQA, and four navigation metrics, and exceeds every source specialist on five 3D scene-understanding benchmarks and R2R navigation. Factorization is nearly lossless versus the uncompressed MoE on 21 of 22 benchmarks; VSI-Bench is the exception, with frame-count experiments suggesting weaker use of cross-frame evidence.

### Source excerpt

arXiv:2611.20767v1 Announce Type: new Abstract: Embodied capabilities such as visual grounding, spatial reasoning, navigation, and driving understanding are often developed in separate vision-language specialists fine-tuned from a common pretrained model.Deploying the specialists separately multiplies storage, whereas merging them into one dense model forces conflicting task updates to share parameters and loses part of each specialist's expertise.We introduce UNITE, a framework that consolidates such specialists into a compact shared-backbone mixture of experts. UNITE keeps the feed-forward block of each specialist as a routable expert and represents the expert bank through a shared-anchor one-bit factorization: a shared floating-point anchor plus expert-specific, row-scaled binary deviations.The anchor is optimized jointly with the scales and signs through closed-form alternating updates, using only source weights and no calibration data.Router adaptation then adjusts token-level expert selection with the backbone and experts frozen, updating fewer than $0.6$M parameters ($<0.01\%$ of the $8.8$B active parameters). Built from four Qwen3-VL-8B specialists, the complete UNITE model occupies $20.3$ GB, $1.16\times$ the storage of one dense specialist and $59.6\%$ below the uncompressed MoE. Across $22$ benchmarks and R2R Val-Unseen, UNITE matches six static merging methods in mean retention of best-specialist performance, leads all of them on Scan2Cap, ScanQA, and all four navigation metrics, and surpasses every source specialist on all five 3D scene understanding benchmarks (up to $+19.1$ on Scan2Cap) and on R2R navigation. Relative to the uncompressed MoE, factorization is nearly lossless on $21$ of $22$ benchmarks; the exception is VSI-Bench, where frame-count experiments suggest impaired use of cross-frame evidence.UNITE thus consolidates complementary embodied capabilities near single-model storage.

## Prepare Lighter and Faster for Open-Vocabulary Queries: A Query-Wise 3D Segmenter for Gaussian Splatting

DevFeed: [Prepare Lighter and Faster for Open-Vocabulary Queries: A Query-Wise 3D Segmenter for Gaussian Splatting](<https://devfeed.tech/articles/prepare-lighter-and-faster-for-open-vocabulary-queries-a-query-wise-3d-segmenter-for-gaussian-splatting-137447.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.19429>)

Author: Nhat-Quynh Le-Pham, Khoi Nguyen, Binh-Son Hua

Published: 2026-11-17T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [gaussian splatting](<https://devfeed.tech/topics/gaussian-splatting.md>), [Machine Perception](<https://devfeed.tech/topics/machine-perception.md>), [3D](<https://devfeed.tech/topics/3d.md>)

Tags: [2d](<https://devfeed.tech/tags/2d.md>), [3d](<https://devfeed.tech/tags/3d.md>), [clip](<https://devfeed.tech/tags/clip.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-gr](<https://devfeed.tech/tags/cs-gr.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [gaussian-splatting](<https://devfeed.tech/tags/gaussian-splatting.md>), [lighter](<https://devfeed.tech/tags/lighter.md>), [lightweight](<https://devfeed.tech/tags/lightweight.md>), [localization](<https://devfeed.tech/tags/localization.md>), [mask](<https://devfeed.tech/tags/mask.md>), [object](<https://devfeed.tech/tags/object.md>), [performance](<https://devfeed.tech/tags/performance.md>), [queries](<https://devfeed.tech/tags/queries.md>), [query](<https://devfeed.tech/tags/query.md>), [text](<https://devfeed.tech/tags/text.md>), [training](<https://devfeed.tech/tags/training.md>), [video](<https://devfeed.tech/tags/video.md>), [visibility](<https://devfeed.tech/tags/visibility.md>), [voting](<https://devfeed.tech/tags/voting.md>)

### AI overview

The framework segments 3D instances in Gaussian Splatting scenes from text queries without per-scene training. It uses dense pixel-level CLIP features for fine-grained localization, selects a target region in a pivot view, propagates it across views with a video segmentor, and maps the resulting 2D tracks to 3D Gaussians through visibility voting. The authors report state-of-the-art results on LERF-OVS, ScanNet, and ScanNet++, competitive results on 3D-OVS, and significant speedups over existing methods. The supplied text is an abstract and does not detail runtime, preprocessing requirements, or the limits of these comparisons.

### Source excerpt

arXiv:2611.19429v1 Announce Type: cross Abstract: Open-vocabulary 3D understanding aims to localize and segment arbitrary objects beyond predefined categories. However, existing approaches based on Gaussian Splatting often rely on expensive preprocessing and lengthy training, limiting their applicability when fast and lightweight solutions are desired. We propose a lightweight, training-free framework for query-driven 3D instance segmentation on Gaussian Splatting. Instead of conventional mask-wise CLIP embeddings, we use dense pixel-level CLIP features to preserve fine-grained semantic information and enable flexible text-driven localization without relying on predefined instance proposals. Given a text query, we identify a high-quality target region in a carefully selected pivot view and propagate it across views using a video segmentor, providing consistent and comprehensive coverage of the target instance. We then lift the resulting 2D tracklets to 3D Gaussians through a novel visibility voting strategy, yielding accurate 3D segmentation masks with clean object boundaries. Extensive experiments demonstrate state-of-the-art performance on LERF-OVS, ScanNet, and ScanNet++, competitive results on 3D-OVS, and significant speedups over existing open-vocabulary 3D segmentation methods. These results highlight the effectiveness of our framework for efficient open-vocabulary 3D understanding without per-scene training.

## Vision-language models lag human performance on physical dynamics and intent reasoning

DevFeed: [Vision-language models lag human performance on physical dynamics and intent reasoning](<https://devfeed.tech/articles/vision-language-models-lag-human-performance-on-physical-dynamics-and-intent-reasoning-134954.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2601.01547>)

Author: Tianjun Gu, Jingyu Gong, Zhizhong Zhang, Yuan Xie, Lizhuang Ma, Xin Tan, Athanasios V. Vasilakos

Published: 2026-11-16T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [vlm](<https://devfeed.tech/topics/vlm.md>), [ai evaluation](<https://devfeed.tech/topics/ai-evaluation.md>), [Machine Perception](<https://devfeed.tech/topics/machine-perception.md>), [LLM evaluation / benchmarking](<https://devfeed.tech/topics/llm-evaluation-benchmarking.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>)

Tags: [benchmark](<https://devfeed.tech/tags/benchmark.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [embodied-ai](<https://devfeed.tech/tags/embodied-ai.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [performance](<https://devfeed.tech/tags/performance.md>), [reasoning](<https://devfeed.tech/tags/reasoning.md>), [spatial-reasoning](<https://devfeed.tech/tags/spatial-reasoning.md>), [vision-language-models](<https://devfeed.tech/tags/vision-language-models.md>)

### AI overview

Vision-language models remain well below first-pass human performance when reasoning jointly about physical dynamics, reference frames, and human intent. The paper introduces Teleo-Spatial Intelligence (TSI) and EscherVerse, a resource built from 11,328 real-world videos, with an 8,000-example benchmark and a 35,963-example instruction-tuning set. Across 27 models, the strongest proprietary model scored 57.26% overall accuracy; first-pass human responses from 11 annotators averaged 90.62% (range 84.81%-95.14%). Fine-tuning open-weight models on real-world, intent-aware data narrowed the gap but did not close it.

### Source excerpt

arXiv:2601.01547v3 Announce Type: replace-cross Abstract: Spatial intelligence is central to embodied cognition, yet contemporary AI systems still struggle to reason about physical interactions in open-world human environments. Despite strong performance on controlled benchmarks, vision-language models often fail to jointly model physical dynamics, reference frames, and the latent human intentions that drive spatial change. We introduce Teleo-Spatial Intelligence (TSI), which captures an important aspect of spatial reasoning: integrating physical dynamics and action goals to reason about actions and their spatial consequences. To evaluate TSI, we present EscherVerse, a large-scale open-world resource built from 11,328 real-world videos, including an 8,000-example benchmark and a 35,963-example instruction-tuning set. Across 27 state-of-the-art vision-language models and an independent analysis of first-pass human responses from 11 annotators, we identify a persistent teleo-spatial reasoning gap: the strongest proprietary model achieves 57.26% overall accuracy, far below first-pass human performance, which ranges from 84.81% to 95.14% with a mean of 90.62%. Fine-tuning on real-world, intent-aware data narrows this gap for open-weight models, but does not close it. EscherVerse provides a diagnostic testbed for purpose-aware spatial reasoning and highlights a critical gap between pattern recognition and human-level understanding in embodied AI.

## EMS-Nav: Embodied Multi-Scale Navigation

DevFeed: [EMS-Nav: Embodied Multi-Scale Navigation](<https://devfeed.tech/articles/ems-nav-embodied-multi-scale-navigation-132148.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.15589>)

Author: Zenghui Yang, Xinan Wang, Danyang Li, Zhe Xu, Zheng Yang, Jingao Xu, Qiang Ma

Published: 2026-11-13T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Robot Navigation](<https://devfeed.tech/topics/robot-navigation.md>), [Machine Perception](<https://devfeed.tech/topics/machine-perception.md>)

Tags: [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-ro](<https://devfeed.tech/tags/cs-ro.md>), [efficiency](<https://devfeed.tech/tags/efficiency.md>), [language](<https://devfeed.tech/tags/language.md>), [mapping](<https://devfeed.tech/tags/mapping.md>), [navigation](<https://devfeed.tech/tags/navigation.md>), [on-device](<https://devfeed.tech/tags/on-device.md>), [robots](<https://devfeed.tech/tags/robots.md>), [scale](<https://devfeed.tech/tags/scale.md>)

### AI overview

EMS-Nav organizes perception and navigation around functionally meaningful regions, grouping objects and their relationships into a hierarchy from local spaces to the broader environment. This is designed to support language goals at different spatial scales while reducing mapping overhead and language-model input size for on-device operation. In reported experiments, it had the highest success rates across object, room, region, and specific-instance targets, and improved overall first-attempt success by 9.39 percentage points over the strongest baseline. The abstract also reports local deployment in real indoor environments; it does not provide further deployment details.

### Source excerpt

arXiv:2611.15589v1 Announce Type: cross Abstract: Industrial inspection robots must reach work areas and specific equipment. These destinations span spatial scales from regions to objects, so robots must relate each target to its surroundings to interpret the task and determine where to go. Existing methods support multi-scale navigation through rich environment representations, but at substantial computational and storage cost. These costs limit deployment where wireless coverage is incomplete and robots must rely on onboard computation to keep navigating. We introduce EMS-Nav, a biologically inspired multi-scale navigation framework for online, on-device operation. Its core innovation is to organize perception, spatial understanding, and navigation around regions, making local spaces the basic units of both environmental understanding and computation. EMS-Nav groups objects and their relationships into functionally meaningful spaces and builds a hierarchy from local areas to the broader environment. This organization supports efficient map construction alongside the interpretation and execution of language goals at different scales. In multi-scale navigation experiments, EMS-Nav achieves the highest success rates across all four target categories: objects, rooms, regions, and specific instances. It outperforms state-of-the-art open-vocabulary mapping and navigation methods, improving overall first-attempt success by 9.39 percentage points over the strongest baseline while reducing mapping overhead and language-model input size. Local deployment in real indoor environments further demonstrates its applicability to on-device embodied navigation. By establishing regions as the organizing basis for perception, spatial understanding, and language-guided action, EMS-Nav provides a system that combines multi-scale understanding with computational efficiency to support diverse language goals on resource-constrained robots.

## Evaluating Video LLMs' Understanding of Corner Cases in Autonomous Driving

DevFeed: [Evaluating Video LLMs' Understanding of Corner Cases in Autonomous Driving](<https://devfeed.tech/articles/evaluating-video-llms-understanding-of-corner-cases-in-autonomous-driving-132283.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.16767>)

Author: Gabriel Manalu, Minh Son Hoang, Hyeontaek Hwang, Jaesung Lee, Hyojin Choi, Tran Tuan Phong, Seon Kwon Kim, Saad Wazir, Daeyoung Kim

Published: 2026-11-13T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [LLM evaluation / benchmarking](<https://devfeed.tech/topics/llm-evaluation-benchmarking.md>), [autonomous vehicles](<https://devfeed.tech/topics/autonomous-vehicles.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Machine Perception](<https://devfeed.tech/topics/machine-perception.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [image recognition](<https://devfeed.tech/topics/image-recognition.md>)

Tags: [autonomous-driving](<https://devfeed.tech/tags/autonomous-driving.md>), [cases](<https://devfeed.tech/tags/cases.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [driving](<https://devfeed.tech/tags/driving.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [video](<https://devfeed.tech/tags/video.md>)

### AI overview

DashBench evaluates 25 Video LLMs on 1,709 curated driving videos, including collision-related and rare non-collision corner cases often missed by benchmarks focused on routine traffic. Using extensive prompts and open-ended questions, the study finds that all evaluated models perform substantially below the stated threshold for safety-critical driving: they often miss or misjudge hazards and hallucinate agents or actions. The abstract describes DashBench as a dataset, evaluation toolkit, and diagnostic resource; it does not provide model-by-model results.

### Source excerpt

arXiv:2611.16767v1 Announce Type: cross Abstract: Natural language explanations promised by modern Video Large Language Models (Video LLMs) offer autonomous driving systems a path toward detailed scene description and interpretable decision making on the road. While recent benchmarks have begun assessing these models on driving scenes, they mainly target routine traffic conditions and general comprehension tasks based on existing driving datasets and therefore fail to evaluate model performance on the rarest and most safety-critical driving events, also known as "corner cases". To provide a comprehensive evaluation of Video LLMs on precisely these scenarios, we introduce DashBench, a benchmark testing 25 Video LLMs on 1,709 carefully curated driving videos spanning not only collision-related corner cases but also rare non-collision ones. Through extensive prompts and open-ended question answering, we expose a critical capability gap: all evaluated models perform substantially below the threshold required for safety-critical driving applications, frequently misjudging or missing dangerous events on the road ahead and hallucinating non-existent agents or actions in their explanations. We believe DashBench, combining an extensive dataset with a rigorous evaluation toolkit and a detailed diagnosis of current model limitations, drives progress toward safer, more explainable autonomous driving systems. The code and data will be shared at: cornercase-dashbench.github.io

## DexRemo: A Force-Sensing Hand Platform and Task Suite for Contact-Rich Removal Manipulation

DevFeed: [DexRemo: A Force-Sensing Hand Platform and Task Suite for Contact-Rich Removal Manipulation](<https://devfeed.tech/articles/dexremo-a-force-sensing-hand-platform-and-task-suite-for-contact-rich-removal-manipulation-133057.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.15981>)

Author: Liman Wang, Hanyang Zhong, Sun Zhaole, Wenting Cao, Babar Jamil, Andy Tyrrell, Shan Luo, Jihong Zhu

Published: 2026-11-13T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Robot Manipulation](<https://devfeed.tech/topics/robot-manipulation.md>), [contact-rich manipulation](<https://devfeed.tech/topics/contact-rich-manipulation.md>), [robot sense of touch](<https://devfeed.tech/topics/robot-sense-of-touch.md>), [Machine Perception](<https://devfeed.tech/topics/machine-perception.md>)

Tags: [abstract](<https://devfeed.tech/tags/abstract.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [axis](<https://devfeed.tech/tags/axis.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [components](<https://devfeed.tech/tags/components.md>), [core](<https://devfeed.tech/tags/core.md>), [coverage](<https://devfeed.tech/tags/coverage.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-ro](<https://devfeed.tech/tags/cs-ro.md>), [easier](<https://devfeed.tech/tags/easier.md>), [embedded](<https://devfeed.tech/tags/embedded.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [evaluations](<https://devfeed.tech/tags/evaluations.md>), [express](<https://devfeed.tech/tags/express.md>), [extension](<https://devfeed.tech/tags/extension.md>), [frame](<https://devfeed.tech/tags/frame.md>), [fuse](<https://devfeed.tech/tags/fuse.md>), [giving](<https://devfeed.tech/tags/giving.md>), [manipulation](<https://devfeed.tech/tags/manipulation.md>), [platform](<https://devfeed.tech/tags/platform.md>), [rich](<https://devfeed.tech/tags/rich.md>), [sensing](<https://devfeed.tech/tags/sensing.md>)

### AI overview

DexRemo pairs a high-force, three-finger robotic hand with modular fingertips and three-axis force sensing, and evaluates removal manipulation across nine real-robot scenarios. Its learned policies combine multi-view vision, proprioception, fingertip pose, and force measurements to predict closed-loop action chunks from demonstrations. Expressing fingertip forces in a shared wrist-centered frame makes force directions consistent across fingers; ablations on six core tasks report improved stability and generalization, especially when wrist-frame forces are combined with fingertip pose, compared with local-frame force inputs or no-force baselines. Three additional scenarios provide held-out coverage.

### Source excerpt

arXiv:2611.15981v1 Announce Type: cross Abstract: Many manipulation tasks require separating adhered, embedded, contaminated, or mechanically constrained targets. We study this setting as removal manipulation, where progress depends on evolving contact states that are only partially observable from vision and kinematics. We introduce DexRemo, a three-finger robotic hand with high-force actuation, modular fingertips, and three-axis fingertip force sensing, together with a nine-scenario real-robot task suite spanning adhesive detachment, surface cleaning and self-cleaning, and occluded retrieval. To make multi-finger force observations easier to fuse, we express fingertip forces in a shared wrist-centered frame, giving force components consistent geometric meaning across fingers and time. Using an Action Chunking Transformer backbone, policies fuse multi-view vision, proprioception, fingertip pose, and force observations to predict closed-loop action chunks from demonstrations. Evaluation uses six core benchmark tasks for controlled fingertip-input and force-frame ablations, plus three extension scenarios for additional held-out coverage. Across these evaluations, the learned policies achieve consistent removal performance, and ablations show that wrist-frame forces, especially with fingertip pose, improve stability and generalization over local-frame force inputs and no-force baselines. Project website: https://dexremo.github.io

## SALA: Semantically Aligned Latent Actions Enable Dual-Axis Scaling of Embodied Foundation Models

DevFeed: [SALA: Semantically Aligned Latent Actions Enable Dual-Axis Scaling of Embodied Foundation Models](<https://devfeed.tech/articles/sala-semantically-aligned-latent-actions-enable-dual-axis-scaling-of-embodied-foundation-models-132313.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.17110>)

Author: Jiachun Jin, Zetong Zhou, Haonan Wang, Xiao Yang, Jun Zhu, Zhijie Deng

Published: 2026-11-13T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [AI Foundation Models](<https://devfeed.tech/topics/ai-foundation-models.md>), [Representation learning](<https://devfeed.tech/topics/representation-learning.md>), [scaling laws](<https://devfeed.tech/topics/scaling-laws.md>), [Machine Perception](<https://devfeed.tech/topics/machine-perception.md>)

Tags: [2](<https://devfeed.tech/tags/2.md>), [abstract](<https://devfeed.tech/tags/abstract.md>), [actions](<https://devfeed.tech/tags/actions.md>), [align](<https://devfeed.tech/tags/align.md>), [appearance](<https://devfeed.tech/tags/appearance.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [axis](<https://devfeed.tech/tags/axis.md>), [bridge](<https://devfeed.tech/tags/bridge.md>), [can](<https://devfeed.tech/tags/can.md>), [cause](<https://devfeed.tech/tags/cause.md>), [cluster](<https://devfeed.tech/tags/cluster.md>), [collect](<https://devfeed.tech/tags/collect.md>), [control](<https://devfeed.tech/tags/control.md>), [costs](<https://devfeed.tech/tags/costs.md>), [cross](<https://devfeed.tech/tags/cross.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-ro](<https://devfeed.tech/tags/cs-ro.md>), [data](<https://devfeed.tech/tags/data.md>), [foundation](<https://devfeed.tech/tags/foundation.md>), [foundation-models](<https://devfeed.tech/tags/foundation-models.md>), [models](<https://devfeed.tech/tags/models.md>), [scaling](<https://devfeed.tech/tags/scaling.md>)

### AI overview

SALA aligns human and robot latent actions using language descriptions of unpaired clips, avoiding the need for paired data. It converts semantic similarities into soft neighborhood structures that guide human latents toward related robot latents, while robot action reconstruction grounds the shared space in control. The authors report two scaling effects: more general human video in pre-training consistently reduces downstream robot action prediction error, and more task-specific human video in post-training substantially improves real-world task success. The supplied text is an abstract and does not provide experimental details or quantify these gains.

### Source excerpt

arXiv:2611.17110v1 Announce Type: cross Abstract: Embodied foundation models hinge on diverse robot interaction data, which are costly to collect. Egocentric human videos offer a complementary source of experience, and latent action models (LAMs) provide a popular means of leveraging these videos for policy supervision. Compared with world-action models (WAMs), LAMs can offer greater training efficiency and potentially lower inference costs by avoiding explicit pixel prediction during policy learning and execution. However, most existing LAMs learn latent actions solely through video reconstruction, which can cause their representations to cluster by visual appearance rather than by the underlying actions. As a result, they struggle to align human actions with their robot counterparts. To bridge the gap, we introduce SALA, which emphasizes establishing the semantic correspondence for latent actions. Without requiring paired data, we use the language descriptions of unpaired human and robot clips to estimate semantic similarities and supervise cross-embodiment correspondence. Specifically, we translate these similarities into soft neighborhood structures that guide human latent actions toward semantically related robot latents, while robot action reconstruction grounds the latent space in control. The resulting shared latent action space provides a unified supervision signal for training an embodied foundation model on both human and robot videos. Notably, latent action supervision enables scaling along two axes: (1) pre-training on increasing amounts of general human video consistently reduces robot action prediction error on downstream tasks, and (2) post-training on increasing amounts of task-specific human video substantially improves real-world robot task success rates.

## Review the Episodes, Refine the Lessons: Simple Memory Evolution for Embodied Agents

DevFeed: [Review the Episodes, Refine the Lessons: Simple Memory Evolution for Embodied Agents](<https://devfeed.tech/articles/review-the-episodes-refine-the-lessons-simple-memory-evolution-for-embodied-agents-125396.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.11509>)

Author: Yifei Yao, Yude Zou, Xing Gao, Guanbo Jiao, Yukai Wang, Jianming Ma, Weinan Zhang, Chunhua Shen

Published: 2026-11-10T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [Machine Perception](<https://devfeed.tech/topics/machine-perception.md>), [Frontier AI](<https://devfeed.tech/topics/frontier-ai.md>), [infini-attention](<https://devfeed.tech/topics/infini-attention.md>)

Tags: [agents](<https://devfeed.tech/tags/agents.md>), [bounded](<https://devfeed.tech/tags/bounded.md>), [completion](<https://devfeed.tech/tags/completion.md>), [control](<https://devfeed.tech/tags/control.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-ro](<https://devfeed.tech/tags/cs-ro.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [evolution](<https://devfeed.tech/tags/evolution.md>), [execution](<https://devfeed.tech/tags/execution.md>), [experience](<https://devfeed.tech/tags/experience.md>), [gpt](<https://devfeed.tech/tags/gpt.md>), [gpt-6-astra](<https://devfeed.tech/tags/gpt-6-astra.md>), [improvement](<https://devfeed.tech/tags/improvement.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [large-language-models](<https://devfeed.tech/tags/large-language-models.md>), [layouts](<https://devfeed.tech/tags/layouts.md>), [memory](<https://devfeed.tech/tags/memory.md>), [multimodal](<https://devfeed.tech/tags/multimodal.md>), [performance](<https://devfeed.tech/tags/performance.md>), [policies](<https://devfeed.tech/tags/policies.md>), [protocol](<https://devfeed.tech/tags/protocol.md>), [recovery](<https://devfeed.tech/tags/recovery.md>)

### AI overview

SimplEvo improves embodied MLLM performance under direct control by retaining experience in self-written text memories, without weight updates, post-trained policies, or privileged information. It builds a coordinate-free task memory from one demonstration, then refines a bounded working memory using training episodes; both are fixed at test time. GPT-6-Astra's EBench success rate rises from 46.7% to 81.7%, and SimplEvo reports 69.63% success and a 75.32 score on RoboDojo. The abstract says text-only memory achieves similar success to memory that also stores visual frames while using fewer tokens. Fixed memories also generalize to unseen layouts and randomized scenes, and GPT-6-Astra's working memory can improve weaker MLLMs without adaptation.

### Source excerpt

arXiv:2611.11509v1 Announce Type: cross Abstract: Recent studies of embodied agents based on frontier multimodal large language models (MLLMs) develop harness engineering around the MLLM, such as harnesses with post-trained policies and recursive self-improvement loops. Yet whether these additions exceed the native capability of the MLLM remains unexplored. We introduce SimplEvo to explore how far an MLLM can improve under direct control when it retains its execution experience in self-written text memories. It requires no weight updates, post-trained policies, or privileged information. The MLLM writes a coordinate-free task memory once from a single demonstration. Under a strict train-test protocol, it then creates a bounded working memory on training episodes only and refines it after each episode with lessons on procedure, completion, and recovery. Both memories stay fixed at test time. On EBench, SimplEvo lifts the success rate (SR) of GPT-6-Astra under direct control from 46.7% to 81.7%, and tops the RoboDojo leaderboard with 69.63% SR and a score of 75.32. Compared with memories that also store past visual frames, SimplEvo reaches a similar SR with text memory alone while using fewer tokens. With its memories fixed, SimplEvo also generalizes to unseen layouts and randomized scene configurations. Moreover, the working memory transfers across models: written by GPT-6-Astra, it can be reused as is by weaker MLLMs and improve their performance without any adaptation of their own.

## GeoMind: A Benchmark for Spatial Understanding in Robotic Manipulation

DevFeed: [GeoMind: A Benchmark for Spatial Understanding in Robotic Manipulation](<https://devfeed.tech/articles/geomind-a-benchmark-for-spatial-understanding-in-robotic-manipulation-126244.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.12175>)

Author: Jinghe Wang, Xinrui Cao, Duo Wu, Chenghao Gu, Yong Zhong, Linjia Kang, Tianyi Xiong, Zhi Wang

Published: 2026-11-10T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Robot Manipulation](<https://devfeed.tech/topics/robot-manipulation.md>), [Benchmark](<https://devfeed.tech/topics/benchmark.md>), [Machine Perception](<https://devfeed.tech/topics/machine-perception.md>)

Tags: [benchmark](<https://devfeed.tech/tags/benchmark.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-ro](<https://devfeed.tech/tags/cs-ro.md>), [evaluations](<https://devfeed.tech/tags/evaluations.md>), [interaction](<https://devfeed.tech/tags/interaction.md>), [language](<https://devfeed.tech/tags/language.md>), [manipulation](<https://devfeed.tech/tags/manipulation.md>), [mapping](<https://devfeed.tech/tags/mapping.md>), [memory](<https://devfeed.tech/tags/memory.md>), [reasoning](<https://devfeed.tech/tags/reasoning.md>), [robotic](<https://devfeed.tech/tags/robotic.md>), [robotic-manipulation](<https://devfeed.tech/tags/robotic-manipulation.md>), [spatial](<https://devfeed.tech/tags/spatial.md>), [spatial-reasoning](<https://devfeed.tech/tags/spatial-reasoning.md>), [tabletop](<https://devfeed.tech/tags/tabletop.md>), [world](<https://devfeed.tech/tags/world.md>)

### AI overview

GeoMind evaluates spatial understanding for tabletop robotic manipulation across 54 task variants covering spatial reasoning and spatial memory. Its pipeline automatically generates and annotates task instances with verified targets and executable interactions. The authors report that diverse embodied agents struggle to track object identities after scenes change and to turn relational or geometric constraints into actions. The abstract frames the benchmark as addressing prior evaluations that use visual cues that can bypass reasoning or focus on static scenes.

### Source excerpt

arXiv:2611.12175v1 Announce Type: cross Abstract: Robotic manipulation has advanced rapidly in mapping visual observations and language instructions to actions. However, reliable manipulation in open-world environments requires robust spatial understanding: the ability to identify targets in complex layouts (spatial reasoning) and adapt to scenes that continuously evolve through interaction (spatial memory). Yet existing embodied benchmarks fall short in evaluating spatial understanding: they often introduce strong visual cues that bypass spatial reasoning, and focus on static scenes that offer little insight into an agent's capacity for spatial memory. To address this gap, we introduce GeoMind, a systematic benchmark for structured spatial understanding through tabletop manipulation. GeoMind comprises 54 evaluated task variants organized around spatial reasoning and spatial memory. It also incorporates a scalable pipeline that automates the generation and annotation of task instances with verified targets and executable interactions. Extensive evaluations across diverse embodied agents reveal their consistent weaknesses in spatial understanding. Our findings highlight that these agents struggle to track identities after spatial changes and convert relational or geometric constraints into executable actions. We release the GeoMind benchmark at https://github.com/Notfound-JH/GEOMIND-benchmark.git.

## UTOPI: Efficient Egocentric Long-Video Understanding in AR via User-Guided Token Pre-Compression

DevFeed: [UTOPI: Efficient Egocentric Long-Video Understanding in AR via User-Guided Token Pre-Compression](<https://devfeed.tech/articles/utopi-efficient-egocentric-long-video-understanding-in-ar-via-user-guided-token-pre-compression-125411.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.11562>)

Author: Ziqi Wang, Su Chen, Qiance Tang, Jieyu Lin, Ziyun Li, Barbara De Salvo, Sai Qian Zhang

Published: 2026-11-10T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Machine Perception](<https://devfeed.tech/topics/machine-perception.md>), [Compression](<https://devfeed.tech/topics/compression.md>), [Neural Network](<https://devfeed.tech/topics/neural-network.md>)

Tags: [ar](<https://devfeed.tech/tags/ar.md>), [attention](<https://devfeed.tech/tags/attention.md>), [augmented-reality](<https://devfeed.tech/tags/augmented-reality.md>), [background](<https://devfeed.tech/tags/background.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [compression](<https://devfeed.tech/tags/compression.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [memory](<https://devfeed.tech/tags/memory.md>), [model](<https://devfeed.tech/tags/model.md>), [motion](<https://devfeed.tech/tags/motion.md>), [neural-network](<https://devfeed.tech/tags/neural-network.md>), [overlap](<https://devfeed.tech/tags/overlap.md>), [performance](<https://devfeed.tech/tags/performance.md>), [platform](<https://devfeed.tech/tags/platform.md>), [pruning](<https://devfeed.tech/tags/pruning.md>), [redundancy](<https://devfeed.tech/tags/redundancy.md>), [retention](<https://devfeed.tech/tags/retention.md>), [sensors](<https://devfeed.tech/tags/sensors.md>), [token](<https://devfeed.tech/tags/token.md>), [user](<https://devfeed.tech/tags/user.md>), [video](<https://devfeed.tech/tags/video.md>)

### AI overview

UTOPI is a plug-and-play pre-compression module for egocentric long videos on resource-constrained AR devices. It uses user motion signals to estimate inter-frame overlap and prune redundant content in a viewpoint-shift-invariant way. Eye-tracking fixations help distinguish attended foreground from background; a fine-tuned neural-network adjustment mechanism preferentially retains attended content. The abstract reports up to 95% compression while preserving performance across multiple benchmarks and model backbones. It does not specify particular hardware requirements, benchmark names, or the exact meaning of "preserving performance."

### Source excerpt

arXiv:2611.11562v1 Announce Type: new Abstract: As long-video understanding gains increasing attention, augmented reality (AR) devices are emerging as a compelling deployment platform. However, egocentric videos exhibit significant temporal redundancy, imposing substantial memory and computational burdens on resource-constrained AR hardware. To address this challenge, we introduce UTOPI, a plug-and-play pre-compression module designed specifically for egocentric long videos on AR platforms. First, we leverage user motion signals to estimate inter-frame overlap and propose a viewpoint-shift-invariant pruning algorithm. Second, we exploit eye-tracking sensors to identify user fixations, enabling the separation of attended foreground content from background and its preferential retention via a fine-tuned neural network-based adjustment mechanism. Across multiple benchmarks and model backbones, UTOPI achieves up to 95% compression while preserving performance. Code and project page are available at https://sai-lab-nyu.github.io/utopi/.

## Spatial Harness: Grounding Spatial Reasoning in Executable World States with Physics Engines

DevFeed: [Spatial Harness: Grounding Spatial Reasoning in Executable World States with Physics Engines](<https://devfeed.tech/articles/spatial-harness-grounding-spatial-reasoning-in-executable-world-states-with-physics-engines-125365.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.11344>)

Author: Chentao Cao, Zhanke Zhou, Sangni Duan, Bo Han, Hang Li

Published: 2026-11-10T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Machine Perception](<https://devfeed.tech/topics/machine-perception.md>), [Computer vision](<https://devfeed.tech/topics/computer-vision.md>), [3D](<https://devfeed.tech/topics/3d.md>), [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>)

Tags: [camera](<https://devfeed.tech/tags/camera.md>), [code](<https://devfeed.tech/tags/code.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [model](<https://devfeed.tech/tags/model.md>), [models](<https://devfeed.tech/tags/models.md>), [physics](<https://devfeed.tech/tags/physics.md>), [qwen3-5](<https://devfeed.tech/tags/qwen3-5.md>), [reasoning](<https://devfeed.tech/tags/reasoning.md>), [spatial](<https://devfeed.tech/tags/spatial.md>), [spatial-reasoning](<https://devfeed.tech/tags/spatial-reasoning.md>), [vision](<https://devfeed.tech/tags/vision.md>), [vision-language-models](<https://devfeed.tech/tags/vision-language-models.md>), [vision-language-models-vlms](<https://devfeed.tech/tags/vision-language-models-vlms.md>), [vlms](<https://devfeed.tech/tags/vlms.md>), [world](<https://devfeed.tech/tags/world.md>)

### AI overview

Spatial Harness grounds spatial reasoning in an executable scene state: a Perceiver combines question-derived constraints with visual estimates to place objects and cameras in a shared coordinate frame, then a Reasoner uses physics-engine measurements and rendered views while executing viewpoint changes. Across four named benchmarks, the system was evaluated with open- and closed-weight VLMs. On MindCube, it raised Qwen3.5-27B accuracy from 52.70% to 71.14%, 7.52 percentage points above pySpatial. With identical annotated object and camera poses, it achieved 98.25%, compared with 71.81% for model-internal reasoning. These figures are reported for the stated MindCube conditions.

### Source excerpt

arXiv:2611.11344v1 Announce Type: new Abstract: Spatial reasoning requires understanding how objects and observers are arranged in a scene and how their relations change with viewpoint or movement. Vision-language models (VLMs) typically perform both steps through model-internal reasoning, forming an implicit scene interpretation from partial views and imagining the consequences of spatial transformations. A misread scene can lead to a wrong answer, while even a correct interpretation does not ensure that imagined transformations satisfy spatial constraints. We introduce Spatial Harness, which represents scenes as explicit and executable world states to ground spatial reasoning in physics-engine feedback. A Perceiver integrates scene constraints extracted from the question with specialized visual estimates to place objects and cameras in a shared coordinate frame. The resulting state is instantiated in a physics engine, where a Reasoner executes viewpoint changes and uses computed measurements and rendered views to guide reasoning. We evaluate on MindCube, SAT, MMSI-Bench, and SpatialRGPT-Bench with open- and closed-weight VLMs. On MindCube, Spatial Harness improves Qwen3.5-27B from 52.70% to 71.14%, surpassing pySpatial by 7.52 percentage points. Given identical annotated object and camera poses, it reaches 98.25% versus 71.81% for model-internal reasoning. Code is publicly available at: https://github.com/tmlr-group/Spatial-Harness

## How to "train" a surgeon? Benchmarking generalist agents in surgical scene understanding

DevFeed: [How to "train" a surgeon? Benchmarking generalist agents in surgical scene understanding](<https://devfeed.tech/articles/how-to-train-a-surgeon-benchmarking-generalist-agents-in-surgical-scene-understanding-126114.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.11437>)

Author: Gengyuan Zhang, Xiao Han, Shijie Zhou, Abdullah Koukash, Jingpei Wu, Jindong Gu, Yuqicheng Zhu, Nassir Navab, Volker Tresp, Ghazal Ghazaei

Published: 2026-11-10T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [AI Chat](<https://devfeed.tech/topics/ai-chat.md>), [Machine Perception](<https://devfeed.tech/topics/machine-perception.md>), [gaia](<https://devfeed.tech/topics/gaia.md>), [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>)

Tags: [agentic](<https://devfeed.tech/tags/agentic.md>), [agents](<https://devfeed.tech/tags/agents.md>), [arxiv](<https://devfeed.tech/tags/arxiv.md>), [assessment](<https://devfeed.tech/tags/assessment.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [benchmarking](<https://devfeed.tech/tags/benchmarking.md>), [bottlenecks](<https://devfeed.tech/tags/bottlenecks.md>), [boundary](<https://devfeed.tech/tags/boundary.md>), [bounded](<https://devfeed.tech/tags/bounded.md>), [challenges](<https://devfeed.tech/tags/challenges.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [data](<https://devfeed.tech/tags/data.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [diagnostic](<https://devfeed.tech/tags/diagnostic.md>), [environment](<https://devfeed.tech/tags/environment.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [failure](<https://devfeed.tech/tags/failure.md>), [fine](<https://devfeed.tech/tags/fine.md>), [generalist](<https://devfeed.tech/tags/generalist.md>), [how-to](<https://devfeed.tech/tags/how-to.md>), [train](<https://devfeed.tech/tags/train.md>)

### AI overview

The study finds that general-purpose multimodal language models have partial visual priors for surgical scenes but struggle with relationships, procedural understanding, and workflow assessment. It introduces SurgKnowBench to evaluate pretrained models across public datasets and SurgAgentGym to test whether bounded SurgWiki search and visual grounding can improve performance at inference time, without additional surgical training. Tool use narrows the gap mainly on perception-heavy tasks; fine-grained localization and expert-style assessment remain bottlenecks. The supplied text is an abstract, so it does not provide detailed methods or results.

### Source excerpt

arXiv:2611.11437v1 Announce Type: cross Abstract: Evaluating general-purpose multimodal large language models (MLLMs) for surgical scene understanding remains limited due to scarce surgical data and fragmented evaluation protocols. We address these challenges through a two-stage evaluation framework for surgical scene understanding. In Stage I, we introduce SurgKnowBench, a diagnostic benchmark that harmonizes public surgical datasets across procedures and tasks and evaluates proprietary and open-source MLLMs to map the boundary of pretrained surgical competence. SurgKnowBench reveals that current models retain partial visual priors but remain weak on relational and procedural understanding and workflow assessment. Building on this failure frontier and motivated by the scaffolded nature of surgical training, we introduce SurgAgentGym, an agentic environment where general-purpose MLLM agents can use bounded knowledge search over SurgWiki and general-purpose visual grounding for surgical tasks. SurgAgentGym evaluates whether generalist agents can improve surgical understanding at inference time through tool-mediated evidence use, without additional surgical training. Our results show that agentic inference narrows the gap mainly for perception-heavy tasks, while fine-grained localization and expert-style assessment remain persistent bottlenecks. This opens a path toward future studies of agentic surgical understanding.

## From Radio Measurements to Sensing Services: A 3GPP Perspective on Multi-UAV Tracking with Networked ISAC

DevFeed: [From Radio Measurements to Sensing Services: A 3GPP Perspective on Multi-UAV Tracking with Networked ISAC](<https://devfeed.tech/articles/from-radio-measurements-to-sensing-services-a-3gpp-perspective-on-multi-uav-tracking-with-networked-isac-126786.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.10355>)

Author: Xingqin Lin

Published: 2026-11-10T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Networking](<https://devfeed.tech/sources/arxiv-networking.md>)

Topics: [5G](<https://devfeed.tech/topics/5g.md>), [communications](<https://devfeed.tech/topics/communications.md>), [Machine Perception](<https://devfeed.tech/topics/machine-perception.md>)

Tags: [acquisition](<https://devfeed.tech/tags/acquisition.md>), [aging](<https://devfeed.tech/tags/aging.md>), [application](<https://devfeed.tech/tags/application.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [case-study](<https://devfeed.tech/tags/case-study.md>), [context](<https://devfeed.tech/tags/context.md>), [coverage](<https://devfeed.tech/tags/coverage.md>), [cs-ni](<https://devfeed.tech/tags/cs-ni.md>), [measurement](<https://devfeed.tech/tags/measurement.md>), [measurements](<https://devfeed.tech/tags/measurements.md>), [mission](<https://devfeed.tech/tags/mission.md>), [network](<https://devfeed.tech/tags/network.md>), [network-architecture](<https://devfeed.tech/tags/network-architecture.md>), [perspective](<https://devfeed.tech/tags/perspective.md>), [position](<https://devfeed.tech/tags/position.md>), [radio](<https://devfeed.tech/tags/radio.md>), [ray](<https://devfeed.tech/tags/ray.md>), [receiver](<https://devfeed.tech/tags/receiver.md>), [sensing](<https://devfeed.tech/tags/sensing.md>), [services](<https://devfeed.tech/tags/services.md>), [tracking](<https://devfeed.tech/tags/tracking.md>)

### AI overview

Accurate position estimates alone do not ensure useful multi-UAV tracking: missed targets, delayed acquisition, track interruptions and aging observations also affect what an application receives. From a 3GPP perspective, the article links sensing-service requirements and network architecture to receiver search regions, measurement context and track validity. A five-UAV case study in a 3GPP urban scenario and a ray-traced San Francisco scene examines the tradeoff between target coverage and extra outputs, acquisition versus ongoing continuity, and testing a static mission envelope before adding adaptation. It also illustrates that a state predicted to the present may still rest on old observations. The abstract's lessons are to align receiver search with requested coverage, retain measurement time and uncertainty, and assess coverage, ambiguity and timeliness together.

### Source excerpt

arXiv:2611.10355v1 Announce Type: new Abstract: Integrated sensing and communication (ISAC) can turn radio measurements into networked sensing services, but accurate position estimates alone do not ensure useful tracking. Missed targets, late acquisition, track interruptions and aging observations also shape the information available to an application. This article examines tracking of noncooperative unmanned aerial vehicles (UAVs) from a Third Generation Partnership Project (3GPP) perspective. It connects sensing-service requirements and network architecture to three information interfaces: receiver search regions, measurement context and track validity. A five-UAV case study in a 3GPP urban scenario and a ray-traced San Francisco scene illustrates the tradeoff between target coverage and extra outputs, the distinction between acquisition and ongoing continuity, and the value of testing a static mission envelope before adding adaptation. An observation-age example explains how a state predicted to the present can still rely on old evidence. The resulting lessons are to align receiver search with requested coverage, preserve measurement time and uncertainty, and assess coverage, ambiguity and timeliness together. We conclude by pointing out fruitful avenues for further work in sensing design, calibration and service validation.

[Next page](<https://devfeed.tech/topics/machine-perception.md?cursor=WyIyMDI2LTExLTEwVDA1OjAwOjAwKzAwOjAwIiwgIjc0ZmE2NWZlLTVhYjItNDU4MC05Mzg2LTFmOWE3OTU2YjIyNiJd>)