# Robot Manipulation

Robot manipulation is a robotics research area focused on enabling robots to interact with and manipulate objects, using methods including machine learning, visual perception, and tactile sensing.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## Reference-Free Humanoid Skateboarding via Termination-Shaped Reinforcement Learning

DevFeed: [Reference-Free Humanoid Skateboarding via Termination-Shaped Reinforcement Learning](<https://devfeed.tech/articles/reference-free-humanoid-skateboarding-via-termination-shaped-reinforcement-learning-145131.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.26054>)

Author: Ganeshvar Soenessardien, Michael Przystupa, Kevin Sebastian Luck

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Humanoid Robots](<https://devfeed.tech/topics/humanoid-robots.md>), [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>), [Robot Manipulation](<https://devfeed.tech/topics/robot-manipulation.md>)

Tags: [board](<https://devfeed.tech/tags/board.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-ro](<https://devfeed.tech/tags/cs-ro.md>), [deep-reinforcement-learning](<https://devfeed.tech/tags/deep-reinforcement-learning.md>), [humanoid](<https://devfeed.tech/tags/humanoid.md>), [interaction](<https://devfeed.tech/tags/interaction.md>), [model](<https://devfeed.tech/tags/model.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>)

### AI overview

The paper presents model-free deep reinforcement learning for teaching the K1 humanoid to skateboard without demonstrations or analytic steering models requiring system identification. It combines task rewards with termination constraints that block behaviors including losing the planted board foot, putting both feet on the board, excessive twisting, separating from the board, and unstable board motion. As these behaviors become less prevalent during training, the constraints produce an implicit progression from stance control toward sustained robot-board locomotion.

### Source excerpt

arXiv:2611.26054v1 Announce Type: cross Abstract: Humanoid locomotion research has advanced rapidly on walking and running, but \emph {object-mediated} locomotion, in which a robot must coordinate its body with a passive external object, remains difficult. One challenging task is skateboarding, which requires simultaneous interaction with the ground and a passive but movable platform to achieve forward motion. Current methods for tackling skateboarding with humanoid robots either require human demonstrations or utilise analytic steering models that require system identification. In this paper, we introduce a deep reinforcement learning framework that does not rely on prior data such as demonstrations and enables robots such as the K1 humanoid to learn to skateboard using model-free learning. The method combines task rewards with exploit-blocking terminations. We introduce a number of constraints restricting behaviours such as losing the planted board foot, standing with both feet on the board, excessive twisting, separation from the board, and unstable board motion. Their changing prevalence during training produces an implicit progression from stance control towards sustained robot--board locomotion.

## TipDex-VTLA: Learning When to Refine Actions with Touch for Dexterous Manipulation

DevFeed: [TipDex-VTLA: Learning When to Refine Actions with Touch for Dexterous Manipulation](<https://devfeed.tech/articles/tipdex-vtla-learning-when-to-refine-actions-with-touch-for-dexterous-manipulation-145571.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.25683>)

Author: Xudi Ge, Yepeng Liu, Haifeng Liu, Enze Shi, Liangqi Zhang, Yongchao Xu

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Robot Manipulation](<https://devfeed.tech/topics/robot-manipulation.md>), [robot sense of touch](<https://devfeed.tech/topics/robot-sense-of-touch.md>)

Tags: [benchmark](<https://devfeed.tech/tags/benchmark.md>), [code](<https://devfeed.tech/tags/code.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-ro](<https://devfeed.tech/tags/cs-ro.md>), [drift](<https://devfeed.tech/tags/drift.md>), [execution](<https://devfeed.tech/tags/execution.md>), [feature](<https://devfeed.tech/tags/feature.md>), [feedback](<https://devfeed.tech/tags/feedback.md>), [forecast](<https://devfeed.tech/tags/forecast.md>), [gate](<https://devfeed.tech/tags/gate.md>), [language](<https://devfeed.tech/tags/language.md>), [learning](<https://devfeed.tech/tags/learning.md>), [lighting](<https://devfeed.tech/tags/lighting.md>), [manipulation](<https://devfeed.tech/tags/manipulation.md>), [policies](<https://devfeed.tech/tags/policies.md>), [real-world](<https://devfeed.tech/tags/real-world.md>), [reference](<https://devfeed.tech/tags/reference.md>), [reliability](<https://devfeed.tech/tags/reliability.md>), [simulation](<https://devfeed.tech/tags/simulation.md>), [source-code](<https://devfeed.tech/tags/source-code.md>), [state](<https://devfeed.tech/tags/state.md>), [tasks](<https://devfeed.tech/tags/tasks.md>)

### AI overview

TipDex-VTLA aims to reduce action drift when lighting changes by deciding when tactile feedback should refine dexterous manipulation actions. Its VASE component estimates visual reliability from a frozen reference policy's sensitivity to illumination changes; TACE compares predicted and measured touch at matched times, then combines the discrepancy with visual reliability to gate bounded feature corrections for actions not yet executed. Trained on data collected under normal visual conditions, it achieved average full-task success rates of 87.5% under normal lighting and 73.8% under challenging unseen lighting across four real-world tasks, and was also evaluated on the UniVTAC simulation benchmark. The excerpt reports outperforming some state-of-the-art methods but gives no specific comparator results.

### Source excerpt

arXiv:2611.25683v1 Announce Type: cross Abstract: Changes in illumination can induce action drift in dexterous manipulation even when the physical state remains unchanged. Although existing vision--tactile--language--action (VTLA) policies incorporate tactile feedback, when and how touch refines actions under changing visual conditions remains challenging. We introduce \textbf{TipDex-VTLA}, a VTLA policy that combines estimated visual action reliability and tactile forecast verification to regulate execution-time refinement. Specifically, Visual Action Sensitivity Estimation (VASE) learns visual reliability from a frozen reference policy's action sensitivity to illumination perturbations. Tactile Anticipation and Correction during Execution (TACE) compares predicted and measured touch at matched times and combines the resulting discrepancy with visual reliability to gate bounded feature corrections for unexecuted actions. This effectively helps to refine actions under challenging unseen visual conditions. We evaluate TipDex-VTLA on four real-world dexterous manipulation tasks and the UniVTAC simulation benchmark. TipDex-VTLA trained on data collected under normal visual conditions, achieves average full-task success rates of 87.5% (resp. 73.8%) under normal (resp. challenging unseen) lighting in real-world tasks, significantly outperforming some state-of-the-art methods. We will release the collected data and source code.

## PGX: Physically Grounded Execution for Long-Horizon Humanoid Loco-Manipulation

DevFeed: [PGX: Physically Grounded Execution for Long-Horizon Humanoid Loco-Manipulation](<https://devfeed.tech/articles/pgx-physically-grounded-execution-for-long-horizon-humanoid-loco-manipulation-145692.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.26596>)

Author: Kai Lu, Chenyang Ma, Ioannis Havoutis

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Humanoid Robots](<https://devfeed.tech/topics/humanoid-robots.md>), [Robot Manipulation](<https://devfeed.tech/topics/robot-manipulation.md>), [Robotics](<https://devfeed.tech/topics/robotics.md>), [Robot Navigation](<https://devfeed.tech/topics/robot-navigation.md>)

Tags: [actions](<https://devfeed.tech/tags/actions.md>), [base](<https://devfeed.tech/tags/base.md>), [based](<https://devfeed.tech/tags/based.md>), [components](<https://devfeed.tech/tags/components.md>), [consistency](<https://devfeed.tech/tags/consistency.md>), [control](<https://devfeed.tech/tags/control.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-ro](<https://devfeed.tech/tags/cs-ro.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [execution](<https://devfeed.tech/tags/execution.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [flat](<https://devfeed.tech/tags/flat.md>), [focus](<https://devfeed.tech/tags/focus.md>), [framework](<https://devfeed.tech/tags/framework.md>), [gate](<https://devfeed.tech/tags/gate.md>), [grounded](<https://devfeed.tech/tags/grounded.md>), [horizon](<https://devfeed.tech/tags/horizon.md>), [humanoid](<https://devfeed.tech/tags/humanoid.md>), [humanoid-robot](<https://devfeed.tech/tags/humanoid-robot.md>), [improvements](<https://devfeed.tech/tags/improvements.md>), [language](<https://devfeed.tech/tags/language.md>), [manipulation](<https://devfeed.tech/tags/manipulation.md>)

### AI overview

PGX addresses long-horizon humanoid loco-manipulation by separating task progression and spatial precision from VLA skill execution. A predicate-based state machine gates phase changes, triggers recovery, and coordinates navigation and waiting; shared predicate semantics also link demonstration segmentation, execution, and stage-wise evaluation. Within each active manipulation phase, flat or stage-trained VLA models focus on relevant actions. Training-free world-coordinate locking maintains spatial consistency through base station keeping, object-relative positioning, and world-anchored contact targets. Experiments on three simulated tasks and one real-world task with a Unitree G1 report improved stage progression and task success rates; the excerpt provides no quantitative results.

### Source excerpt

arXiv:2611.26596v1 Announce Type: cross Abstract: Robotic vision-language-action (VLA) models can execute diverse individual manipulation skills, yet they still struggle with long-horizon loco-manipulation tasks, where temporal structure, skills, and precision are tightly coupled within a monolithic model. We propose Physically Grounded Execution (PGX), a framework that offloads temporal and precision control to two classical training-free components, allowing the VLA to focus on stage-specific skill dynamics. PGX decomposes long-horizon loco-manipulation into three layers: (i) a predicate-based state machine governs task progression, using physical predicates to gate phase transitions, trigger recovery, and coordinate navigation and waiting. Shared predicate semantics connect offline demonstration segmentation, online execution, and stage-wise evaluation; (ii) VLA skills operate within active manipulation phases, with flat or stage-trained models concentrating on task-relevant actions; and (iii) world-coordinate locking (WCL) maintains spatial consistency through base station keeping, object-relative positioning, and world-anchored contact targets. We demonstrate the effectiveness of PGX with extensive experiments conducted on three simulated tasks and one real-world task using a Unitree G1 humanoid robot for long-horizon loco-manipulation, showing improvements in stage progression and task success rates. Project page is available at: https://kl-research.github.io/pgx

## VG-Steer: Visual-Geometric Policy Steering from In-Context Aligned Abstractions

DevFeed: [VG-Steer: Visual-Geometric Policy Steering from In-Context Aligned Abstractions](<https://devfeed.tech/articles/vg-steer-visual-geometric-policy-steering-from-in-context-aligned-abstractions-145481.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.25199>)

Author: Arthur Bucker, Pramod Anantharam, Jonathan Francis, Jean Oh

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Robot Manipulation](<https://devfeed.tech/topics/robot-manipulation.md>), [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>)

Tags: [abstractions](<https://devfeed.tech/tags/abstractions.md>), [adapting](<https://devfeed.tech/tags/adapting.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [bottleneck](<https://devfeed.tech/tags/bottleneck.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-ro](<https://devfeed.tech/tags/cs-ro.md>), [data](<https://devfeed.tech/tags/data.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [distribution](<https://devfeed.tech/tags/distribution.md>), [efficiently](<https://devfeed.tech/tags/efficiently.md>), [flow-matching](<https://devfeed.tech/tags/flow-matching.md>), [inference](<https://devfeed.tech/tags/inference.md>), [manipulation](<https://devfeed.tech/tags/manipulation.md>), [ood](<https://devfeed.tech/tags/ood.md>), [pairs](<https://devfeed.tech/tags/pairs.md>), [policies](<https://devfeed.tech/tags/policies.md>), [policy](<https://devfeed.tech/tags/policy.md>), [real-world](<https://devfeed.tech/tags/real-world.md>), [reasoning](<https://devfeed.tech/tags/reasoning.md>), [region](<https://devfeed.tech/tags/region.md>), [robot](<https://devfeed.tech/tags/robot.md>), [steering](<https://devfeed.tech/tags/steering.md>)

### AI overview

VG-Steer adapts frozen robot policies at inference by transferring motion patterns from successful in-distribution rollouts. It represents those patterns as object-relational distance pairs, then combines task-relevant entities and geometric projections using Large Vision Model semantics and Vision-Language Model reasoning to produce visual-geometric guidance. The abstract reports efficient correction of flow-matching policies under pose and goal shifts in simulation. In a real-robot experiment, it more than doubled success on cable placements one grid step beyond training positions, expanded workspace coverage, and steered a single-task policy toward unseen objects while nearly eliminating object confusion. These are abstract-level claims; details of the experiments and limitations are not provided.

### Source excerpt

arXiv:2611.25199v1 Announce Type: cross Abstract: Adapting robot policies to out-of-distribution (OOD) settings with limited real-world data remains a key bottleneck to generalist robots. The multimodality and scarcity of robotics data lead end-to-end approaches to overfit and break under small distribution shifts. Test-time adaptation can alleviate this by correcting frozen policies at inference, but most methods ground their correction in a signal learned implicitly from robotic datasets, making it susceptible to the same shifts as the policy it corrects. Recent methods that instead exploit explicit spatial information are typically limited to coarse corrections, missing the nuanced, task-relevant motion most manipulation tasks require. We propose VG-Steer, which corrects existing policies by transferring the nuanced motion of successful in-distribution rollouts, captured along meaningful geometric axes, known here as object-relational distance pairs. VG-Steer leverages the semantic understanding of Large Vision Models (LVMs) and Vision-Language Models (VLMs) reasoning, which combines the task-relevant entity pairs and their object-relational distance projections into a single tailored visual-geometric guidance signal. In a simulation benchmark, it efficiently corrects flow-matching policies under pose and task-goal shifts; on a real robot experiment, it widens the region of the workspace the frozen policy covers, more than doubling its success rate on cable placements one grid step outside the training positions, and steers a single-task policy toward objects it was never trained on while nearly eliminating object confusion. Project website: https://arthurfenderbucker.github.io/VG-Steer/

## Awaken-HI: Scaling Humanoid Intelligence with Whole-Body Human Experience

DevFeed: [Awaken-HI: Scaling Humanoid Intelligence with Whole-Body Human Experience](<https://devfeed.tech/articles/awaken-hi-scaling-humanoid-intelligence-with-whole-body-human-experience-145669.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.26420>)

Author: EngineAI

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Humanoid Robots](<https://devfeed.tech/topics/humanoid-robots.md>), [Robot Manipulation](<https://devfeed.tech/topics/robot-manipulation.md>), [Robotics](<https://devfeed.tech/topics/robotics.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>)

Tags: [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-ro](<https://devfeed.tech/tags/cs-ro.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [humanoid](<https://devfeed.tech/tags/humanoid.md>), [humanoid-robotics](<https://devfeed.tech/tags/humanoid-robotics.md>), [law](<https://devfeed.tech/tags/law.md>), [manipulation](<https://devfeed.tech/tags/manipulation.md>), [model](<https://devfeed.tech/tags/model.md>), [motion](<https://devfeed.tech/tags/motion.md>), [people](<https://devfeed.tech/tags/people.md>), [pretraining](<https://devfeed.tech/tags/pretraining.md>), [scale](<https://devfeed.tech/tags/scale.md>), [tasks](<https://devfeed.tech/tags/tasks.md>), [vision](<https://devfeed.tech/tags/vision.md>), [vision-language-action](<https://devfeed.tech/tags/vision-language-action.md>), [whole-body-motion](<https://devfeed.tech/tags/whole-body-motion.md>)

### AI overview

Awaken-Human pairs about 50,000 hours of egocentric stereo video with synchronized whole-body motion across more than 100 skills. Using this data, Awaken-HI jointly predicts human motion and robot actions through a shared action expert; directly supervising that expert with human trajectories improves transfer to humanoid loco-manipulation. The authors report that more human pretraining data lowers held-out robot action prediction error and raises downstream real-robot success. After full-corpus pretraining and T800 adaptation, the model reaches a 93.75% mean success rate across eight individual tasks, outperforming GR00T N1.7 and π₀.₅ in individual and long-horizon evaluations. It also generalizes to objects not seen during robot post-training and new pairings of familiar objects and destinations.

### Source excerpt

arXiv:2611.26420v1 Announce Type: cross Abstract: A central goal of humanoid robotics is for robots to perform the everyday work people do in homes and workplaces. Existing egocentric video datasets cover many of these activities but provide little synchronized whole-body motion data for learning loco-manipulation at scale. We build Awaken-Human, which pairs approximately 50,000 hours of egocentric stereo video with synchronized whole-body human motion and covers more than 100 skills. Building on this, we also introduce Awaken-HI (Humanoid Intelligence), a vision-language-action model for humanoid loco-manipulation that jointly predicts original human motion and retargeted robot actions through a shared action expert. We show that loco-manipulation skills transfer more effectively from people to humanoids when human trajectories directly supervise the shared action expert. Increasing human pretraining data reduces held-out robot action prediction error according to an empirical scaling law and raises the mean success rate on downstream real-robot tasks. After full-corpus pretraining and T800 adaptation, Awaken-HI achieves a mean success rate of 93.75% across eight individual tasks and outperforms GR00T N1.7 and $\pi_{0.5}$ in both individual and long-horizon evaluations. The same policy generalizes to objects unseen during robot post-training and to new pairings of familiar objects and destinations.

## MAWAM: A Multi-Agent World-Action Model for Coordinated Manipulation

DevFeed: [MAWAM: A Multi-Agent World-Action Model for Coordinated Manipulation](<https://devfeed.tech/articles/mawam-a-multi-agent-world-action-model-for-coordinated-manipulation-145467.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.25105>)

Author: Xianzhe Fan, Chi-Pin Huang, Fu-En Yang, Jing Wang, Yecheng Wu, Yu-Cheng Chou, Johan Bjorck, Linxi "Jim" Fan, Jan Kautz, Yu-Chiang Frank Wang, Zhiding Yu, Hengshuang Zhao, Zhiqi Li

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Multi Agent Systems](<https://devfeed.tech/topics/multi-agent-systems.md>), [Robot Manipulation](<https://devfeed.tech/topics/robot-manipulation.md>), [World models](<https://devfeed.tech/topics/world-models.md>), [Physical AI](<https://devfeed.tech/topics/physical-ai.md>)

Tags: [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-ro](<https://devfeed.tech/tags/cs-ro.md>), [manipulation](<https://devfeed.tech/tags/manipulation.md>), [model](<https://devfeed.tech/tags/model.md>), [multi-agent](<https://devfeed.tech/tags/multi-agent.md>), [robotics](<https://devfeed.tech/tags/robotics.md>)

### AI overview

MAWAM predicts a robot team's physical future alongside its actions: it jointly generates each robot's future views and executable action chunks, using cross-agent attention while keeping separate control interfaces. On three real-robot tasks, it reports 67.8% mean success, 18.9 percentage points above MA-VLA trained on the same demonstrations and cameras. In simulation, it reports 90.3% mean success across eight tasks with teams of up to four arms. Ablations indicate both cross-agent attention and team-stage future-video supervision matter: removing attention reduced four-arm relay success from 95.0% to 39.0%, while removing that video supervision reduced Place Food success from 78.0% to 47.0%.

### Source excerpt

arXiv:2611.25105v1 Announce Type: cross Abstract: Every move in a robot team changes the world in which its teammates must act. Coordinated manipulation therefore calls for predicting the team's actions together with the physical future they jointly create. We introduce MAWAM, a multi-agent world-action model that jointly generates each robot's future views and executable action chunks in a single sample. Built from a pretrained single-robot world-action backbone, MAWAM jointly denoises all robots' action and future-view tokens through cross-agent attention, while preserving a separate control interface for each robot. MAWAM achieves the highest success on all three real-robot tasks (laundry-bag collection, bath-towel folding, and umbrella handover), with 67.8% mean success, 18.9 percentage points above MA-VLA trained on the same demonstrations and cameras. In simulation, MAWAM reaches 90.3% mean success on eight tasks with teams of up to four arms. Ablations show that cross-agent attention and team-stage future-video supervision both improve coordinated control: without attention, a four-arm relay falls from 95.0% to 39.0%, and without this supervision, Place Food falls from 78.0% to 47.0%.

## GroundingPI: A Grounding Foundation Model towards Physical Intelligence with Visual Primitives

DevFeed: [GroundingPI: A Grounding Foundation Model towards Physical Intelligence with Visual Primitives](<https://devfeed.tech/articles/groundingpi-a-grounding-foundation-model-towards-physical-intelligence-with-visual-primitives-144951.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2609.39601>)

Author: Qize Yu, Lianrui Fan, Boyu Chen, Jiaqi Liang, Xini Ding, Yue Chen, Zetian Song, Yuran Wang, Yi Zou, Kaixuan Wang, Tianxing Chen, Wenxuan Song, Bohan Zhou, Mingleyang Li, Siqiao Huang, Yuqi Ye, Caigao

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [AI Foundation Models](<https://devfeed.tech/topics/ai-foundation-models.md>), [Physical AI](<https://devfeed.tech/topics/physical-ai.md>), [Robot Manipulation](<https://devfeed.tech/topics/robot-manipulation.md>), [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>), [grpo](<https://devfeed.tech/topics/grpo.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [scaling laws](<https://devfeed.tech/topics/scaling-laws.md>)

Tags: [autonomous-driving](<https://devfeed.tech/tags/autonomous-driving.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-ro](<https://devfeed.tech/tags/cs-ro.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [foundation-model](<https://devfeed.tech/tags/foundation-model.md>), [grounding](<https://devfeed.tech/tags/grounding.md>), [grpo](<https://devfeed.tech/tags/grpo.md>), [multimodal](<https://devfeed.tech/tags/multimodal.md>), [ocr](<https://devfeed.tech/tags/ocr.md>), [pretraining](<https://devfeed.tech/tags/pretraining.md>), [robotic-manipulation](<https://devfeed.tech/tags/robotic-manipulation.md>)

### AI overview

GroundingPI is a 4B model that represents points and boxes as quantized coordinates in a shared vocabulary, trained with multimodal and spatial pretraining, supervised fine-tuning, and GRPO reinforcement learning. The abstract reports an average of 73.68% across 34 grounding benchmarks, ahead of the larger GPT-6 Astra at 71.54%. As a visual backbone, it outperformed evaluated alternatives in all four out-of-distribution settings across RoboTwin 2.0 and RoboCasa-GR1, with gains up to 24.8%; on RoboCasa-GR1, using 50% of demonstrations beat baselines using 75%. Its mean open-loop L2 error on nuScenes was 0.296 m. The authors report that greater pretraining scale improved grounding and manipulation, dense grounding benefited both, and OCR may aid perceptual learning.

### Source excerpt

arXiv:2609.39601v3 Announce Type: replace-cross Abstract: Precise grounding matters. It specifies which object is the target and where that object is, even in clutter and for tiny objects, and it has to be fast enough for closed-loop control. Yet vision-language-action (VLA) and world-action models (WAMs) take perception from general-purpose vision-language and video-generation backbones, which still fail in these settings. We introduce GroundingPI, a 4B grounding foundation model that generates points and boxes as quantized coordinates in a shared vocabulary. Training combines multimodal and spatial pretraining, supervised fine-tuning, and reinforcement learning with GRPO, using supervision from public datasets and dedicated data engines. Against 44 baselines across 34 grounding benchmarks spanning 11 perceptual capabilities, GroundingPI establishes a new state of the art, averaging 73.68%, above the larger GPT-6 Astra (71.54%). As a downstream visual backbone, GroundingPI improves performance on robotic manipulation and autonomous driving. Across RoboTwin 2.0 and RoboCasa-GR1, it outperforms all evaluated backbones in all four out-of-distribution settings, with relative gains of up to 24.8% over the strongest baseline. On RoboCasa-GR1, GroundingPI trained with 50% of the demonstrations outperforms those baselines trained with 75%. Its mean open-loop L2 error on nuScenes is 0.296 m. We systematically analyze GroundingPI's pretraining in scale and data composition. Grounding quality and downstream manipulation improve with pretraining scale, with generalization tracking perceptual gains most closely. Analyzing the data recipe across these 11 perceptual capabilities shows dense grounding's substantial benefits for both, and OCR's potential as a catalyst for perceptual learning. These results support grounding as a perceptual foundation, and dedicated perceptual pretraining as a promising direction for foundation models of physical intelligence.

## Learning Object-Centric Spatial Reasoning for Sequential Manipulation in Cluttered Environments

DevFeed: [Learning Object-Centric Spatial Reasoning for Sequential Manipulation in Cluttered Environments](<https://devfeed.tech/articles/learning-object-centric-spatial-reasoning-for-sequential-manipulation-in-cluttered-environments-145318.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2603.02511>)

Author: Chrisantus Eze, Ryan C Julian, Christopher Crick

Published: 2026-11-20T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Robot Manipulation](<https://devfeed.tech/topics/robot-manipulation.md>), [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>)

Tags: [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-ro](<https://devfeed.tech/tags/cs-ro.md>), [learning](<https://devfeed.tech/tags/learning.md>), [manipulation](<https://devfeed.tech/tags/manipulation.md>), [object](<https://devfeed.tech/tags/object.md>), [reasoning](<https://devfeed.tech/tags/reasoning.md>), [spatial](<https://devfeed.tech/tags/spatial.md>), [spatial-reasoning](<https://devfeed.tech/tags/spatial-reasoning.md>)

### AI overview

Unveiler separates object selection from action execution using a discrete object-index interface: its Spatial Relationship Encoder (SRE) selects an object, and a separate Action Decoder generates orientation-discretized push-grasp actions. The SRE learns first from heuristic demonstrations, then improves through search using simulated object removals; this requires no real-robot training data and permits pairing it with another execution policy. In physical Dofbot-Pro experiments, it retrieved two occluded targets in the prescribed order in 90% of scenes, versus 20-30% for the heuristic, GPT-4o, and imitation-only variant. In PyBullet, it led across six clutter and occlusion conditions, achieving 90.0% completion with full occlusion and 6-9 objects, versus 67.5% for the heuristic and 60.0% for ThinkGrasp.

### Source excerpt

arXiv:2603.02511v2 Announce Type: replace-cross Abstract: Sequential manipulation in clutter requires selecting which objects to remove and determining how to manipulate them. Learning both decisions jointly can complicate training and make failures difficult to diagnose. We present Unveiler, a modular framework that separates scene reasoning and object selection from action execution through a discrete object-index interface. Its Spatial Relationship Encoder (SRE) selects the best object to grasp, while a separate Action Decoder generates orientation-discretized push-grasp actions. The SRE is trained in two stages: imitation learning from heuristic demonstrations, followed by search-based policy improvement using simulated object removals. This training procedure requires no real-robot training data and allows the SRE to be paired with a different execution policy. We evaluate Unveiler in PyBullet simulation and on a physical Dofbot-Pro. In real-robot experiments, it retrieves two occluded targets in the prescribed order in 90\% of scenes, compared with 20-30\% for the heuristic, GPT-4o, and the imitation-only variant. In PyBullet, Unveiler achieves the highest task-completion rate across all six clutter-density and occlusion conditions, reaching 90.0\% under full occlusion with 6-9 objects, compared with 67.5\% for the heuristic and 60.0\% for ThinkGrasp. These results demonstrate the potential of modular, object-centric spatial reasoning for sequential manipulation in clutter.

## Teaching robots to carve marble sculptures

DevFeed: [Teaching robots to carve marble sculptures](<https://devfeed.tech/articles/teaching-robots-to-carve-marble-sculptures-143718.md>)

Original publisher: [Read original article](<https://blog.adafruit.com/2026/11/19/teaching-robots-to-carve-marble-sculptures/>)

Author: Anne Barela

Published: 2026-11-19T15:05:20Z

Content type: article

Language: en

Sources: [Adafruit Industries](<https://devfeed.tech/sources/adafruit-industries-makers-hackers-artists-designers-and-engineers.md>)

Topics: [Robot Manipulation](<https://devfeed.tech/topics/robot-manipulation.md>), [3D](<https://devfeed.tech/topics/3d.md>)

Tags: [3d](<https://devfeed.tech/tags/3d.md>), [3d-model](<https://devfeed.tech/tags/3d-model.md>), [art](<https://devfeed.tech/tags/art.md>), [artificial-intelligence](<https://devfeed.tech/tags/artificial-intelligence.md>), [block](<https://devfeed.tech/tags/block.md>), [brooklyn](<https://devfeed.tech/tags/brooklyn.md>), [demo](<https://devfeed.tech/tags/demo.md>), [embodied-ai](<https://devfeed.tech/tags/embodied-ai.md>), [history](<https://devfeed.tech/tags/history.md>), [marble](<https://devfeed.tech/tags/marble.md>), [nyc](<https://devfeed.tech/tags/nyc.md>), [robotics](<https://devfeed.tech/tags/robotics.md>), [robots](<https://devfeed.tech/tags/robots.md>), [sculpture](<https://devfeed.tech/tags/sculpture.md>), [system](<https://devfeed.tech/tags/system.md>)

### AI overview

Monumental Labs uses KUKA KR240 robots to turn a 3D model and marble block into a sculpture without a person operating the machine. The software represents the stone as a sparse voxel grid, derives a signed distance field, and plans roughing passes to remove bulk material, including tilted passes for overhangs. Finishing removes the remaining stone in offset layers, with regional toolpaths following geodesic distances along the surface. It also calculates collision-free tool orientations and joint angles and simulates the full run before machining begins--a practical safeguard when the block is costly to machine.

### Source excerpt

Monumental Labs in Greenpoint, Brooklyn has put KUKA KR240 robots to work carving marble, and the team presented the system at Embodied AI NYC. It takes a 3D model and a block of stone and produces a finished sculpture without a person running the machine. The demo piece was a magic trick in marble: a [...]

## Robots can do backflips but can't hold a gaze or press piano keys, and scientists think they've fixed that

DevFeed: [Robots can do backflips but can't hold a gaze or press piano keys, and scientists think they've fixed that](<https://devfeed.tech/articles/robots-can-do-backflips-but-can-t-hold-a-gaze-or-press-piano-keys-and-scientists-think-they-ve-fixed-that-143166.md>)

Original publisher: [Read original article](<https://www.techspot.com/news/114610-robots-can-do-backflips-but-cant-hold-gaze.html>)

Author: Skye Jacobs

Published: 2026-11-19T11:51:00Z

Content type: news

Language: en

Sources: [TechSpot](<https://devfeed.tech/sources/techspot.md>)

Topics: [Robot Manipulation](<https://devfeed.tech/topics/robot-manipulation.md>), [Physical AI](<https://devfeed.tech/topics/physical-ai.md>), [Humanoid Robots](<https://devfeed.tech/topics/humanoid-robots.md>), [3D](<https://devfeed.tech/topics/3d.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [computer-vision](<https://devfeed.tech/tags/computer-vision.md>), [easy](<https://devfeed.tech/tags/easy.md>), [embedded](<https://devfeed.tech/tags/embedded.md>), [engineer](<https://devfeed.tech/tags/engineer.md>), [engineers](<https://devfeed.tech/tags/engineers.md>), [explore](<https://devfeed.tech/tags/explore.md>), [eye](<https://devfeed.tech/tags/eye.md>), [eyeballs](<https://devfeed.tech/tags/eyeballs.md>), [eyes](<https://devfeed.tech/tags/eyes.md>), [feedback](<https://devfeed.tech/tags/feedback.md>), [fiber](<https://devfeed.tech/tags/fiber.md>), [flexible](<https://devfeed.tech/tags/flexible.md>), [folding](<https://devfeed.tech/tags/folding.md>), [gauge](<https://devfeed.tech/tags/gauge.md>), [graphene](<https://devfeed.tech/tags/graphene.md>), [groups](<https://devfeed.tech/tags/groups.md>), [hand](<https://devfeed.tech/tags/hand.md>), [hardware](<https://devfeed.tech/tags/hardware.md>), [materials](<https://devfeed.tech/tags/materials.md>), [muscular](<https://devfeed.tech/tags/muscular.md>), [research](<https://devfeed.tech/tags/research.md>), [robots](<https://devfeed.tech/tags/robots.md>), [science](<https://devfeed.tech/tags/science.md>)

### AI overview

University of Pennsylvania engineers built a soft artificial muscle fiber that can hold a precise position by sensing its own temperature and correcting heat loss. A liquid crystal elastomer contracts when heated, graphene makes it softer and tougher, and an embedded copper coil serves as both heater and temperature-feedback sensor. The system adjusts heating in real time to counter the fiber's tendency to cool and drift back, without a separate sensor. In lab demonstrations, a robot hand varied the force and speed of piano-key presses, while paired fibers moved robotic eyes with controlled, repeatable gaze. The hand was not a skilled pianist. The researchers' next step is combining many fibers into larger systems; coordinating them with AI computer vision is a longer-term goal.

### Source excerpt

The work, published in Advanced Functional Materials, is modeled on the small muscles that move human fingers and eyes. Those muscles make fine, controlled movements, and that kind of control has been hard to get from the hardware inside most robots today. Read Entire Article

## WorldComposer: Generative High-Fidelity Simulation with Digital Cousins for Generalizable Robot Learning and Evaluation

DevFeed: [WorldComposer: Generative High-Fidelity Simulation with Digital Cousins for Generalizable Robot Learning and Evaluation](<https://devfeed.tech/articles/worldcomposer-generative-high-fidelity-simulation-with-digital-cousins-for-generalizable-robot-learning-and-evaluation-142517.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2604.15805>)

Author: Jasper Lu, Zhenhao Shen, Yuanfei Wang, Shugao Liu, Shengqiang Xu, Shawn Xie, Jingkai Xu, Feng Jiang, Jade Yang, Chen Xie, Ruihai Wu

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Robotics Simulation](<https://devfeed.tech/topics/robotics-simulation.md>), [Robot Manipulation](<https://devfeed.tech/topics/robot-manipulation.md>), [Dataset development](<https://devfeed.tech/topics/dataset-development.md>)

Tags: [automated](<https://devfeed.tech/tags/automated.md>), [correlation](<https://devfeed.tech/tags/correlation.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-ro](<https://devfeed.tech/tags/cs-ro.md>), [data-augmentation](<https://devfeed.tech/tags/data-augmentation.md>), [engines](<https://devfeed.tech/tags/engines.md>), [environments](<https://devfeed.tech/tags/environments.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [framework](<https://devfeed.tech/tags/framework.md>), [generation](<https://devfeed.tech/tags/generation.md>), [generative](<https://devfeed.tech/tags/generative.md>), [interactive](<https://devfeed.tech/tags/interactive.md>), [learning](<https://devfeed.tech/tags/learning.md>), [manipulation](<https://devfeed.tech/tags/manipulation.md>), [maps](<https://devfeed.tech/tags/maps.md>), [object](<https://devfeed.tech/tags/object.md>), [physics](<https://devfeed.tech/tags/physics.md>), [real-world](<https://devfeed.tech/tags/real-world.md>), [robot](<https://devfeed.tech/tags/robot.md>), [sim-to-real](<https://devfeed.tech/tags/sim-to-real.md>), [simulation](<https://devfeed.tech/tags/simulation.md>), [tasks](<https://devfeed.tech/tags/tasks.md>)

### AI overview

WorldComposer maps real-world panoramas into task-ready simulations and generates "Digital Cousins" by varying scenes and objects, aiming to expand robot-learning data without repeatedly acquiring physical assets or reconfiguring environments. The framework combines realistic assets and physics engines for manipulation of rigid, articulated, and deformable objects, and stitches rooms into larger environments for long-horizon tasks. The abstract reports strong sim-to-real correlation and improved generalization to unseen scene and object variations as generated data scales up.

### Source excerpt

arXiv:2604.15805v2 Announce Type: replace-cross Abstract: Learning robust robot policies in real-world environments requires diverse data augmentation, yet scaling real-world data collection is costly due to the need for acquiring physical assets and reconfiguring environments. Therefore, augmenting real-world scenes into simulation has become a practical augmentation for efficient learning and evaluation. We present WorldComposer, an automated generative Real2Sim2Real framework that maps real-world panoramas into high-fidelity, task-ready simulation environments and further synthesizes diverse Digital Cousins through scene and object variations. Combined with high-quality physics engines and realistic assets, WorldComposer supports interactive manipulation across rigid, articulated, and deformable objects. Additionally, we incorporate multi-room stitching to construct consistent large-scale environments for long-horizon tasks. Experiments demonstrate a strong sim-to-real correlation validating our platform's fidelity, and show that extensively scaling up data generation leads to significantly better generalization to unseen scene and object variations, demonstrating the effectiveness of Digital Cousins for generalizable robot learning and evaluation.

## Finetuning VLAs for Dexterous Manipulation: The Right Checkpoint Is All You Need

DevFeed: [Finetuning VLAs for Dexterous Manipulation: The Right Checkpoint Is All You Need](<https://devfeed.tech/articles/finetuning-vlas-for-dexterous-manipulation-the-right-checkpoint-is-all-you-need-143430.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.23676>)

Author: Artur Kuramshin, Charlotte Cloutier, Michael Przystupa, Kirsty Ellis, Mohammad S. Akhter, Glen Berseth

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Robot Manipulation](<https://devfeed.tech/topics/robot-manipulation.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [lora](<https://devfeed.tech/topics/lora.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>)

Tags: [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-ro](<https://devfeed.tech/tags/cs-ro.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [diffusion](<https://devfeed.tech/tags/diffusion.md>), [finetuning](<https://devfeed.tech/tags/finetuning.md>), [lora](<https://devfeed.tech/tags/lora.md>), [manipulation](<https://devfeed.tech/tags/manipulation.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [sft](<https://devfeed.tech/tags/sft.md>), [simulation](<https://devfeed.tech/tags/simulation.md>), [steering](<https://devfeed.tech/tags/steering.md>), [study](<https://devfeed.tech/tags/study.md>), [supervised-fine-tuning](<https://devfeed.tech/tags/supervised-fine-tuning.md>), [vision-language-action](<https://devfeed.tech/tags/vision-language-action.md>), [vision-language-model](<https://devfeed.tech/tags/vision-language-model.md>), [vla](<https://devfeed.tech/tags/vla.md>)

### AI overview

Using π₀.₅ and a LEAP hand, the study compares five VLA fine-tuning recipes on two simulation tasks with a handful of demonstrations. SFT rollout success poorly predicts which checkpoint will perform best after diffusion-steering reinforcement learning (DSRL). Freezing the vision-language model and fully tuning the action expert yields the most steerable policy, including for out-of-distribution placements; LoRA is least steerable. The authors recommend selecting checkpoints without relying on SFT rollout success. These findings are reported for the study's two simulated tasks.

### Source excerpt

arXiv:2611.23676v1 Announce Type: cross Abstract: Vision-Language-Action models have demonstrated strong manipulation capabilities. A limitation of VLA models is that most manipulation datasets focus on parallel-jaw gripper systems, making transfer to multi-finger hands challenging, particularly as teleoperation datasets are harder to collect with multi-fingered robots. In this work, we study how to best adapt VLAs to dexterous hands by pairing supervised fine-tuning (SFT) with diffusion-steering reinforcement learning (DSRL). Using $\pi_{0.5}$ and a LEAP hand, we evaluate five fine-tuning recipes across two simulation tasks using only a handful of demonstrations. Our empirical analysis yields three key insights: (1) SFT success is poorly correlated with post-DSRL success, meaning the best SFT checkpoint is rarely the most effective to steer; (2) freezing the vision-language model (VLM) and tuning only the action expert produces the most steerable policy for both training and out-of-distribution placements; and (3) low-rank adaptation (LoRA) is the least steerable approach. Consequently, we recommend an adaptation recipe that freezes the VLM, fully trains the action expert, and eschews SFT rollout success as a checkpoint selection metric.

## CriticHack: Evaluating Visual Rewards Under Robot Policy Optimization

DevFeed: [CriticHack: Evaluating Visual Rewards Under Robot Policy Optimization](<https://devfeed.tech/articles/critichack-evaluating-visual-rewards-under-robot-policy-optimization-142571.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.02527>)

Author: Jiaxuan Luo, Xingguo Xu, Shanshan Wang, Yuhan Zhou, Zhen Zhang

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Robot Manipulation](<https://devfeed.tech/topics/robot-manipulation.md>), [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>)

Tags: [camera](<https://devfeed.tech/tags/camera.md>), [completion](<https://devfeed.tech/tags/completion.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-ro](<https://devfeed.tech/tags/cs-ro.md>), [diffusion](<https://devfeed.tech/tags/diffusion.md>), [drawer](<https://devfeed.tech/tags/drawer.md>), [execution](<https://devfeed.tech/tags/execution.md>), [exploit](<https://devfeed.tech/tags/exploit.md>), [failure](<https://devfeed.tech/tags/failure.md>), [model](<https://devfeed.tech/tags/model.md>), [object](<https://devfeed.tech/tags/object.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [policy](<https://devfeed.tech/tags/policy.md>), [progress](<https://devfeed.tech/tags/progress.md>), [repair](<https://devfeed.tech/tags/repair.md>), [reward](<https://devfeed.tech/tags/reward.md>), [reward-model](<https://devfeed.tech/tags/reward-model.md>), [robot](<https://devfeed.tech/tags/robot.md>), [simulator](<https://devfeed.tech/tags/simulator.md>), [stacking](<https://devfeed.tech/tags/stacking.md>), [training](<https://devfeed.tech/tags/training.md>), [trust](<https://devfeed.tech/tags/trust.md>)

### AI overview

Fine-tuning a robot policy against visual rewards can improve task success while also increasing wrong-object failures. In drawer opening, Robometer training raised success by 10.2 percentage points and wrong-drawer failures by 10.9; RoboReward-8B showed the same pattern, which also appeared on a physical robot. The proposed mechanism is that wrong-object actions look like progress and score better than many ordinary failures, so optimization amplifies them even when successes score higher. The authors suggest a pre-training check using initial-policy executions to identify outcomes that score unusually well relative to alternatives. They also report a label-free repair: rescore executions on frames re-rendered with unnamed objects restored, which removed the amplification in the tested tasks and robot setting. In a stacking task, the check predicted which placement would be exploited: wrong-object stacks rose from 20% to 69% only for that placement.

### Source excerpt

arXiv:2610.02527v2 Announce Type: replace-cross Abstract: Learned visual reward models score a robot's execution from camera images and an instruction, and are increasingly used to fine-tune robot policies. Two checks decide whether to trust one: the reward model separates successes from failures, and task success rises during training. We show that both checks can pass while the policy learns to act on the wrong object. When a diffusion policy is fine-tuned against the reward model Robometer to open a drawer, task success rises by 10.2 percentage points. Wrong-object failures, which open a different drawer, rise by 10.9 points, whereas the same training on the simulator's task-completion signal raises success alone. A second reward model, RoboReward-8B, behaves the same way, and the effect reappears on a physical robot. This happens because of how the reward scores the policy's own failures. Most failures make little progress and score low, while a wrong-object failure looks like progress and scores well above them. Optimization shifts a policy toward outcomes that score above the executions they compete with, the executions they can take the place of. It therefore turns low-scoring failures into both successes and wrong-object failures, even when the reward model scores successes higher. This rule needs only executions of the initial policy, so it can be tested before training. On a stacking task we predicted which of three object placements Robometer would exploit, and training raised wrong-object stacks from 20% to 69% in that placement and in neither of the others. The same rule gives a check to run before optimization and a repair that needs no labels: scoring each execution again on frames re-rendered with the objects the instruction does not name put back removes the amplification on both tasks and on the robot.

## RECLAIM: Robot-Vision-Informed Adversarial Patch Defense via Embodied Attention Feedback

DevFeed: [RECLAIM: Robot-Vision-Informed Adversarial Patch Defense via Embodied Attention Feedback](<https://devfeed.tech/articles/reclaim-robot-vision-informed-adversarial-patch-defense-via-embodied-attention-feedback-143156.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.24494>)

Author: Jiayou Zhu, Peng Guo, Litian Zhang, Xinze Shi, Yuting Zhang, Zelong Yu, Yize Tian, Jirong Liu, Zifeng Kang

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Cryptography and Security](<https://devfeed.tech/sources/arxiv-cryptography-and-security.md>)

Topics: [Adversarial attacks](<https://devfeed.tech/topics/adversarial-attacks.md>), [Robot Manipulation](<https://devfeed.tech/topics/robot-manipulation.md>), [Machine Perception](<https://devfeed.tech/topics/machine-perception.md>)

Tags: [actions](<https://devfeed.tech/tags/actions.md>), [adversarial](<https://devfeed.tech/tags/adversarial.md>), [agent](<https://devfeed.tech/tags/agent.md>), [attack](<https://devfeed.tech/tags/attack.md>), [attention](<https://devfeed.tech/tags/attention.md>), [changes](<https://devfeed.tech/tags/changes.md>), [control](<https://devfeed.tech/tags/control.md>), [cs-cr](<https://devfeed.tech/tags/cs-cr.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-ro](<https://devfeed.tech/tags/cs-ro.md>), [decision](<https://devfeed.tech/tags/decision.md>), [defense](<https://devfeed.tech/tags/defense.md>), [edits](<https://devfeed.tech/tags/edits.md>), [effective](<https://devfeed.tech/tags/effective.md>), [feedback](<https://devfeed.tech/tags/feedback.md>), [localization](<https://devfeed.tech/tags/localization.md>), [manipulation](<https://devfeed.tech/tags/manipulation.md>), [models](<https://devfeed.tech/tags/models.md>), [multiple](<https://devfeed.tech/tags/multiple.md>), [patch](<https://devfeed.tech/tags/patch.md>), [patches](<https://devfeed.tech/tags/patches.md>), [robot](<https://devfeed.tech/tags/robot.md>), [vision](<https://devfeed.tech/tags/vision.md>)

### AI overview

RECLAIM is a training-free, inference-time defense that uses a frozen vision-language-action policy's attention response to local image edits to guide patch localization and purification during robot manipulation. It combines that feedback with robot geometry, then revalidates treatments as actions change the observed scene. In evaluation against patches from four attack methods across multiple task suites, it achieved at least one successful rollout on 95% of evaluated tasks and improved LIBERO-Object rollout success from 0% without defense to 70.0%. Mean decision time was 548.9 ms per control step. The abstract reports these results under frozen VLA control; it does not detail the specific local treatments.

### Source excerpt

arXiv:2611.24494v1 Announce Type: cross Abstract: Vision-language-action policies control embodied agents through a perception-action loop grounded in physical interaction, where adversarial patches can corrupt action predictions and thereby alter subsequent observations. To mitigate these threats, many existing defenses rely on policy adaptation or auxiliary models, but require additional training or inference. We introduce RECLAIM, a training-free, inference-time defense that uses a frozen VLA's own attention response to local visual interventions to guide purification during closed-loop embodied manipulation. Our key insight is that attention changes induced by local edits provide feedback for choosing where and how to purify as the robot interacts with its environment. RECLAIM combines this feedback with robot geometry to refine patch localization and select local treatments, then revalidates them as the agent's embodied actions change the observed scene. We evaluate RECLAIM against patches generated by four adversarial attack methods across multiple manipulation task suites under frozen VLA control. Under adversarial patch attacks, RECLAIM succeeds in at least one rollout on 95% of the evaluated tasks and raises rollout success on LIBERO-Object from 0% without defense to 70.0%. Its mean decision time is only 548.9 ms per control step. These results highlight policy-guided visual purification as an effective approach to defending embodied agents against adversarial patches and recovering closed-loop task execution.

## LAGR: Language-Aware Grasp Re-Ranking for Task Oriented Grasping

DevFeed: [LAGR: Language-Aware Grasp Re-Ranking for Task Oriented Grasping](<https://devfeed.tech/articles/lagr-language-aware-grasp-re-ranking-for-task-oriented-grasping-142712.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.23858>)

Author: Manav Kulshrestha, Shreyans Jain, Apoorva Vashisth, Ankush Gupta, Damon Conover, Aniket Bera

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Robot Manipulation](<https://devfeed.tech/topics/robot-manipulation.md>), [vlm](<https://devfeed.tech/topics/vlm.md>)

Tags: [annotations](<https://devfeed.tech/tags/annotations.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-ro](<https://devfeed.tech/tags/cs-ro.md>), [dataset](<https://devfeed.tech/tags/dataset.md>), [detection](<https://devfeed.tech/tags/detection.md>), [language](<https://devfeed.tech/tags/language.md>), [manipulation](<https://devfeed.tech/tags/manipulation.md>), [ranking](<https://devfeed.tech/tags/ranking.md>), [robotic-manipulation](<https://devfeed.tech/tags/robotic-manipulation.md>), [robotics](<https://devfeed.tech/tags/robotics.md>)

### AI overview

LAGR targets task-oriented grasping by balancing physical feasibility with suitability for a requested task. The authors introduce a dataset containing 2.68 million scored 5-DoF grasp annotations across 100,911 image-task pairs and 17 object categories. Given vision-language inputs, LAGR generates task-independent grasp proposals with feasibility scores, then uses a language-conditioned re-ranker to assess proposals jointly. The abstract reports a 9.3 percentage-point gain over the next-best method and validation through task-oriented grasp executions on real objects with a Franka Research 3 robot.

### Source excerpt

arXiv:2611.23858v1 Announce Type: cross Abstract: Grasping is a fundamental capability for robotic manipulation, with the appropriate grasp depending on the downstream task. For example, a hammer should be held by its handle for hammering but by its head when it is handed over. Selecting such a grasp requires consideration of two criteria: physical feasibility on the object and suitability for the task. Developing such an approach requires dense grasp labels with both feasibility and task scores on real observations, but existing task-oriented grasping datasets are small, synthetic, or costly to annotate. To address this gap, we construct a dataset of 2.68 million scored 5-DoF grasp annotations over 100,911 image-task pairs from $17$ object categories. Additionally, we introduce LAGR, a task-oriented grasp detection method that explicitly addresses both these challenges. Given vision-language inputs, LAGR first generates task-independent grasp proposals along with a physical feasibility score, and then scores the generated grasps with a language-conditioned re-ranker that reasons over all proposals jointly. Experiments show that LAGR outperforms the next best method by 9.3 percentage points. We further validate LAGR on real objects by executing task-oriented grasps with a Franka Research 3 robot.

## Pose6DAug: Physically Plausible Multi-view Object Swapping for Robot Data Augmentation

DevFeed: [Pose6DAug: Physically Plausible Multi-view Object Swapping for Robot Data Augmentation](<https://devfeed.tech/articles/pose6daug-physically-plausible-multi-view-object-swapping-for-robot-data-augmentation-143277.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2606.20118>)

Author: Jonghoon Lee, Seong Hyeon Park, Byungwoo Jeon, Minha Lee, Jinwoo Shin

Published: 2026-11-19T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Robot Manipulation](<https://devfeed.tech/topics/robot-manipulation.md>), [data augmentation](<https://devfeed.tech/topics/data-augmentation.md>), [3D](<https://devfeed.tech/topics/3d.md>), [Enhancing AI model performance](<https://devfeed.tech/topics/enhancing-ai-model-performance.md>)

Tags: [2d](<https://devfeed.tech/tags/2d.md>), [3d](<https://devfeed.tech/tags/3d.md>), [appearance](<https://devfeed.tech/tags/appearance.md>), [camera](<https://devfeed.tech/tags/camera.md>), [coherent](<https://devfeed.tech/tags/coherent.md>), [consistency](<https://devfeed.tech/tags/consistency.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-ro](<https://devfeed.tech/tags/cs-ro.md>), [data](<https://devfeed.tech/tags/data.md>), [data-augmentation](<https://devfeed.tech/tags/data-augmentation.md>), [data-collection](<https://devfeed.tech/tags/data-collection.md>), [distribution](<https://devfeed.tech/tags/distribution.md>), [failure](<https://devfeed.tech/tags/failure.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [framework](<https://devfeed.tech/tags/framework.md>), [manipulation](<https://devfeed.tech/tags/manipulation.md>), [mesh](<https://devfeed.tech/tags/mesh.md>), [object](<https://devfeed.tech/tags/object.md>), [performance](<https://devfeed.tech/tags/performance.md>), [plausible](<https://devfeed.tech/tags/plausible.md>), [policies](<https://devfeed.tech/tags/policies.md>), [robot](<https://devfeed.tech/tags/robot.md>)

### AI overview

Pose6DAug creates targeted demonstrations for VLA policy failure cases by swapping the manipulated object in existing successful episodes while preserving their action trajectories. It anchors the replacement object with a mesh and temporally coherent 6D pose trajectory in 3D, enabling geometrically consistent renderings across calibrated camera views. This addresses multi-view inconsistency and implausible edits that can arise from naive 2D video editing, especially with heavy occlusion and egocentric views. The abstract reports a 16.5% success-rate improvement over the state-of-the-art baseline on novel objects while preserving in-distribution performance; on a real robot, success more than doubled relative to the base policy. These are abstract-level claims; the supplied text gives no further experimental details.

### Source excerpt

arXiv:2606.20118v3 Announce Type: replace-cross Abstract: Vision-language-action (VLA) policies have shown strong potential for general-purpose manipulation, yet they often fail on novel, out-of-distribution objects whose appearance or geometry deviates from the training distribution. The standard remedy is to collect multi-view teleoperation data for every failure case, but this scales poorly in both cost and time. We introduce Pose6DAug, a failure-driven data augmentation framework that turns existing successful episodes into targeted demonstrations for a policy's failure modes, without any new data collection. Our key insight is that each successful episode already encodes a physically valid action trajectory together with calibrated multi-view observations. By swapping only the manipulated object while preserving this trajectory, we obtain new and physically grounded demonstrations. However, the challenge lies in synthesizing object swaps, e.g., naive 2D video editing breaks multi-view consistency and physical plausibility, particularly under heavy occlusion and egocentric viewpoints. Our method instead operates directly in 3D, anchoring the target object with an explicit mesh driven by a temporally coherent 6D pose trajectory. This ensures geometrically consistent renderings across all camera views. Fine-tuning a VLA on data augmented by our method improves success rates by 16.5% relative to the state-of-the-art baseline on novel objects, while preserving in-distribution performance. On a real robot, it more than doubles the success rate of the base policy. These results show that multi-view and physically consistent augmentation is a practical path to scalable VLA generalization.

## Task Convergent Flow Matching for Generative Robotics

DevFeed: [Task Convergent Flow Matching for Generative Robotics](<https://devfeed.tech/articles/task-convergent-flow-matching-for-generative-robotics-141794.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.22609>)

Author: Lorenzo Paiola, Maximilian St\\"olzle, Giorgio Grioli, Daniela Rus, Cosimo Della Santina

Published: 2026-11-18T05:00:00Z

Content type: news

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Robot Manipulation](<https://devfeed.tech/topics/robot-manipulation.md>), [contact-rich manipulation](<https://devfeed.tech/topics/contact-rich-manipulation.md>), [generative-ai-tools](<https://devfeed.tech/topics/generative-ai-tools.md>)

Tags: [continuous](<https://devfeed.tech/tags/continuous.md>), [convergence](<https://devfeed.tech/tags/convergence.md>), [covariance](<https://devfeed.tech/tags/covariance.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-ro](<https://devfeed.tech/tags/cs-ro.md>), [distribution](<https://devfeed.tech/tags/distribution.md>), [flow-matching](<https://devfeed.tech/tags/flow-matching.md>), [funnel](<https://devfeed.tech/tags/funnel.md>), [generative](<https://devfeed.tech/tags/generative.md>), [hybrid](<https://devfeed.tech/tags/hybrid.md>), [inference](<https://devfeed.tech/tags/inference.md>), [manipulation](<https://devfeed.tech/tags/manipulation.md>), [motion](<https://devfeed.tech/tags/motion.md>), [parallel](<https://devfeed.tech/tags/parallel.md>), [performance](<https://devfeed.tech/tags/performance.md>), [physics](<https://devfeed.tech/tags/physics.md>), [progress](<https://devfeed.tech/tags/progress.md>), [quantile](<https://devfeed.tech/tags/quantile.md>), [robotics](<https://devfeed.tech/tags/robotics.md>), [search](<https://devfeed.tech/tags/search.md>), [source](<https://devfeed.tech/tags/source.md>), [stacking](<https://devfeed.tech/tags/stacking.md>)

### AI overview

The framework refines flow-matching robot policies at inference time by testing candidate action chunks in parallel physics-based rollouts and searching for chunks that make task progress. Performance funnels allow bounded temporary regressions but require net progress; interval-wise Lipschitz bounds are used to verify continuous-time funnel satisfaction from sampled rollouts. For contact-rich dynamics, CMA-ES searches source seeds within radial quantile bands to help retain the learned motion prior. The method supports Euclidean and pose-valued Riemannian Flow Matching policies. Reported success rises from 31.3% to 88.5% on PushT and from 77.0% to 86.0% on three-cube stacking. On PushT, a learned funnel reduces average maximum loss of achieved progress by 19% versus checking chunk endpoints alone, at similar success. Radial-band search yields 11.3% out-of-distribution motion windows, compared with 34.1% for unconstrained search.

### Source excerpt

arXiv:2611.22609v1 Announce Type: cross Abstract: Successful manipulation requires the manipulated objects to progress toward a desired task outcome. Generative policies can produce plausible robot motions without ensuring this progress, allowing successive action chunks to stall or undo earlier changes to the scene. We introduce a task-centric inference-time refinement framework for flow matching policies that evaluates candidate action chunks through parallel physics-based rollouts and searches for motions satisfying sufficient conditions for task convergence. Task-level performance funnels bound temporary regressions while requiring net progress over each chunk. We establish sufficient conditions under which repeated funnel satisfaction yields convergence throughout execution, and use interval-wise Lipschitz bounds to verify continuous-time funnel satisfaction from sampled rollouts. To accommodate contact-rich hybrid dynamics, we refine the flow's source seeds using derivative-free Covariance Matrix Adaptation Evolution Strategy (CMA-ES), constrained to radial quantile bands of the source distribution to help retain the learned motion prior. This formulation supports both Euclidean and pose-valued Riemannian Flow Matching (RMFM) policies. On PushT and three-cube stacking, refinement raises success from 31.3% to 88.5% and from 77.0% to 86.0%. On PushT, whose coverage score responds continuously to the motion of the block, a learned funnel lowers the average maximum loss of achieved progress by 19% relative to constraining the endpoint of each chunk alone, at a similar success rate. With the radial band, 11.3% of the executed motion windows are out of distribution, a rate within the range of held-out demonstrations, compared with 34.1% for an unconstrained search.

## Let the Mind Conceive, Let the Hand Achieve: Coupling Grounded Action and Progress Evaluation for Long-Horizon Manipulation

DevFeed: [Let the Mind Conceive, Let the Hand Achieve: Coupling Grounded Action and Progress Evaluation for Long-Horizon Manipulation](<https://devfeed.tech/articles/let-the-mind-conceive-let-the-hand-achieve-coupling-grounded-action-and-progress-evaluation-for-long-horizon-manipulation-139835.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2610.18352>)

Author: Youpeng Wen, Gang Wang, Tianyu Huang, Weiwei Zhang, Zhijian Huang, Wei Chen, Xieyuanli Chen, Yunhui Liu

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Robot Manipulation](<https://devfeed.tech/topics/robot-manipulation.md>), [Agent Harness](<https://devfeed.tech/topics/agent-harness.md>), [Multi-step AI processes](<https://devfeed.tech/topics/multi-step-ai-processes.md>), [Physical AI](<https://devfeed.tech/topics/physical-ai.md>)

Tags: [agent-harness](<https://devfeed.tech/tags/agent-harness.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-ro](<https://devfeed.tech/tags/cs-ro.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [grounded](<https://devfeed.tech/tags/grounded.md>), [horizon](<https://devfeed.tech/tags/horizon.md>), [manipulation](<https://devfeed.tech/tags/manipulation.md>), [multimodal](<https://devfeed.tech/tags/multimodal.md>), [order](<https://devfeed.tech/tags/order.md>), [planning](<https://devfeed.tech/tags/planning.md>), [policy](<https://devfeed.tech/tags/policy.md>), [progress](<https://devfeed.tech/tags/progress.md>), [robots](<https://devfeed.tech/tags/robots.md>), [tasks](<https://devfeed.tech/tags/tasks.md>)

### AI overview

Orchestra separates planning and execution management from action generation to help robots carry out ordered, long-horizon tasks. It breaks a multimodal instruction into subtasks, tracks completion, updates object and destination locations, and gives an action policy shared context; a separately trained evaluator estimates when to advance. On nine RMBench tasks, Orchestra reached 68.22% mean success, 26.22 percentage points above Mem-0. On RMBench-Compose, its average success length was 7.35 subtasks, 2.46 times HarnessVLA's. Across four real-world multi-step tasks, it improved mean success length by 25% over the same harness using the base π₀.₅ policy. These results are limited to the reported benchmarks and tasks.

### Source excerpt

arXiv:2610.18352v2 Announce Type: replace-cross Abstract: A robot following a multi-step instruction needs to act on the right objects in the right order, even as the scene changes. It must also maintain awareness of task progress and determine when to advance. To address these challenges, we introduce Orchestra, an agent harness that separates task planning and execution management from action generation. Orchestra decomposes a complete multimodal instruction into ordered subtasks, tracks completed steps, and updates the locations of relevant objects and destinations during execution. It maintains a shared context containing the active subtask, spatial grounding, and recent observations. An action policy uses this context to generate actions, while a separately trained progress evaluator estimates when to advance to the next subtask. To evaluate whether robots trained on atomic instructions and short trajectories can execute longer subtask sequences, we introduce a benchmark named RMBench-Compose. Orchestra achieves a mean success rate of $68.22\%$ across nine RMBench tasks, outperforming Mem-0 by $26.22$ percentage points. On RMBench-Compose, it achieves an Average Success Length (ASL) of $7.35$ subtasks, $2.46$ times that of HarnessVLA. Across four real-world multi-step tasks, it improves mean ASL by $25.0\%$ over Orchestra using the base $\pi_{0.5}$ policy. These results show that Orchestra helps robots carry out longer sequences of subtasks while preserving the requested subtask order and acting on the specified objects. Our code and project page are available at https://youpengwen.github.io/Orchestra

## HubEgo-100K: In-the-Wild Egocentric Video of Skilled Work for Robot Learning

DevFeed: [HubEgo-100K: In-the-Wild Egocentric Video of Skilled Work for Robot Learning](<https://devfeed.tech/articles/hubego-100k-in-the-wild-egocentric-video-of-skilled-work-for-robot-learning-141703.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.21938>)

Author: Javad Amirian, Armin Kiani, Lo\\"ic Schaeffer, Tim Sprecher, Mohamed Taha, Karthik Yadati

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Robot Manipulation](<https://devfeed.tech/topics/robot-manipulation.md>), [Dataset development](<https://devfeed.tech/topics/dataset-development.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [multimodal-ai](<https://devfeed.tech/topics/multimodal-ai.md>), [lerobot](<https://devfeed.tech/topics/lerobot.md>)

Tags: [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [camera](<https://devfeed.tech/tags/camera.md>), [cc](<https://devfeed.tech/tags/cc.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-ro](<https://devfeed.tech/tags/cs-ro.md>), [data](<https://devfeed.tech/tags/data.md>), [dataset](<https://devfeed.tech/tags/dataset.md>), [diversity](<https://devfeed.tech/tags/diversity.md>), [environment](<https://devfeed.tech/tags/environment.md>), [environments](<https://devfeed.tech/tags/environments.md>), [imu](<https://devfeed.tech/tags/imu.md>), [kitchen](<https://devfeed.tech/tags/kitchen.md>), [learning](<https://devfeed.tech/tags/learning.md>), [lerobot](<https://devfeed.tech/tags/lerobot.md>), [privacy](<https://devfeed.tech/tags/privacy.md>), [robot](<https://devfeed.tech/tags/robot.md>), [video](<https://devfeed.tech/tags/video.md>), [work](<https://devfeed.tech/tags/work.md>), [workplace](<https://devfeed.tech/tags/workplace.md>)

### AI overview

HubEgo-100K v1.0 releases 10,124 hours of 1080p egocentric skilled-work video, recorded by 12,227 contributors across 15,548 environments, with synchronized 100 Hz IMU data from a head-mounted phone camera. Its 72,754 episodes span 11 setting types, 30 work categories, 56 task types and 35 action families; 83% of hours are workplace recordings. Episodes include machine-generated action and language annotations and are available in LeRobot and MCAP formats under CC BY 4.0. The authors measure diversity across these dimensions and report that new contributors and words continue to appear as randomly ordered episodes are added. They relate dataset properties, including hand visibility and inertial sensing, to published evidence on six properties tied to robot policy performance. This version documents collection, curation and privacy measures, but does not yet include the benchmarks or later releases planned for the project.

### Source excerpt

arXiv:2611.21938v1 Announce Type: cross Abstract: Robot policies trained by imitation generalize poorly outside the environments and objects seen in training. Published data-scaling studies find that generalization grows with the diversity of training environments and objects rather than with more demonstrations per environment. Robot data is costly to collect across many environments, whereas egocentric human video can be recorded in the wild at scale; existing egocentric video, however, mostly shows daily-life activity and contains little skilled work in real workplaces. We present HubEgo-100K, a dataset built for diversity. This version of the paper covers the first release, v1.0: 10,124 hours of 1080p egocentric video with synchronized 100 Hz inertial (IMU) data from one head-mounted phone camera. In v1.0, 12,227 contributors record their own unscripted work in 15,548 environments (each one contributor's own setting of one type, such as their kitchen or workshop) across 11 setting types, 30 work categories, 56 task types and 35 action families, with 83% of hours in workplace settings. The release comprises 72,754 episodes with machine-generated action and language annotations, in LeRobot and MCAP formats under CC BY 4.0. We measure diversity along each axis, show that inside v1.0, with episodes in random order, new contributors and words keep appearing, and relate the measurements to published evidence on six data properties tied to policy performance, including hand visibility and inertial sensing. We also document collection, curation, privacy measures and data format, and provide a datasheet. Later versions of this paper will add benchmarks and cover further releases toward 100,000 hours.

## ALOHA Lightning: Learning Fast and Precise Manipulation

DevFeed: [ALOHA Lightning: Learning Fast and Precise Manipulation](<https://devfeed.tech/articles/aloha-lightning-learning-fast-and-precise-manipulation-139927.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.21919>)

Author: John Hua Yao. Qi Wu, Yihuai Gao, Chelsea Finn, Zipeng Fu

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>), [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Robot Manipulation](<https://devfeed.tech/topics/robot-manipulation.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>)

Tags: [battery](<https://devfeed.tech/tags/battery.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-ro](<https://devfeed.tech/tags/cs-ro.md>), [cs-sy](<https://devfeed.tech/tags/cs-sy.md>), [data-collection](<https://devfeed.tech/tags/data-collection.md>), [deployment](<https://devfeed.tech/tags/deployment.md>), [eess-sy](<https://devfeed.tech/tags/eess-sy.md>), [execution](<https://devfeed.tech/tags/execution.md>), [fast](<https://devfeed.tech/tags/fast.md>), [flash](<https://devfeed.tech/tags/flash.md>), [learning](<https://devfeed.tech/tags/learning.md>), [lightning](<https://devfeed.tech/tags/lightning.md>), [manipulation](<https://devfeed.tech/tags/manipulation.md>), [masking](<https://devfeed.tech/tags/masking.md>), [pipeline](<https://devfeed.tech/tags/pipeline.md>), [platform](<https://devfeed.tech/tags/platform.md>), [policies](<https://devfeed.tech/tags/policies.md>), [robotic-manipulation](<https://devfeed.tech/tags/robotic-manipulation.md>), [robots](<https://devfeed.tech/tags/robots.md>), [skills](<https://devfeed.tech/tags/skills.md>), [speed](<https://devfeed.tech/tags/speed.md>), [system](<https://devfeed.tech/tags/system.md>), [tables](<https://devfeed.tech/tags/tables.md>), [tasks](<https://devfeed.tech/tags/tasks.md>)

### AI overview

ALOHA Lightning (Flash ALOHA) aims to bring learned bimanual manipulation closer to human speed by pairing kinesthetic teaching on a backdrivable platform with a learning pipeline for fast execution. The pipeline uses test-time action smoothing and masking to align visual data between collection and deployment. With 50 demonstrations per task, the system reportedly achieved over 80% success on tasks including ball tossing and catching, folding shorts, battery insertion, and bussing tables, at close to human speed.

### Source excerpt

arXiv:2611.21919v1 Announce Type: cross Abstract: Learning from human demonstrations has enabled robots to acquire a wide range of manipulation skills, but learned policies typically execute far slower than ordinary humans. This speed gap is mainly due to lack of an interface for collecting demonstration data at high speed, and the difficulty in training policies that can robustly execute high-speed motions. In this paper, we present ALOHA Lightning (Flash ALOHA), a system for learning fast and precise robotic manipulation. Our system uses kinesthetic teaching to intuitively collect near-human-speed demonstrations on a backdrivable bimanual platform, yielding natural and fast trajectories. We also present a learning pipeline that enables smooth high-speed execution through test-time action smoothing and aligns the visual data distribution between data collection and deployment with masking. Given 50 demonstrations for each task, ALOHA Lightning autonomously completes bimanual tasks such as ball tossing and catching, folding shorts, battery insertion, and bussing tables for over 80% success rates while close to human speed. Project website: https://flash-aloha.github.io/

## Heave-Ho: Decentralized Long-Horizon Multi-Humanoid Collaboration

DevFeed: [Heave-Ho: Decentralized Long-Horizon Multi-Humanoid Collaboration](<https://devfeed.tech/articles/heave-ho-decentralized-long-horizon-multi-humanoid-collaboration-141759.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.22275>)

Author: Ting-Hsuan Chen, Siheng Zhao, Han Jiang, Qi Pan, Louison Lu, Jiaheng Luo, Henghui Bao, Yiyu Chen, Quan Nguyen, Yue Wang

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Humanoid Robots](<https://devfeed.tech/topics/humanoid-robots.md>), [Robot Manipulation](<https://devfeed.tech/topics/robot-manipulation.md>), [Physical AI](<https://devfeed.tech/topics/physical-ai.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [bench](<https://devfeed.tech/tags/bench.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [collaboration](<https://devfeed.tech/tags/collaboration.md>), [communication](<https://devfeed.tech/tags/communication.md>), [control](<https://devfeed.tech/tags/control.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [cs-ro](<https://devfeed.tech/tags/cs-ro.md>), [decentralized](<https://devfeed.tech/tags/decentralized.md>), [design](<https://devfeed.tech/tags/design.md>), [develop](<https://devfeed.tech/tags/develop.md>), [evaluations](<https://devfeed.tech/tags/evaluations.md>), [execution](<https://devfeed.tech/tags/execution.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [follower](<https://devfeed.tech/tags/follower.md>), [foundation](<https://devfeed.tech/tags/foundation.md>), [framework](<https://devfeed.tech/tags/framework.md>), [horizon](<https://devfeed.tech/tags/horizon.md>), [humanoid](<https://devfeed.tech/tags/humanoid.md>), [humanoids](<https://devfeed.tech/tags/humanoids.md>), [language](<https://devfeed.tech/tags/language.md>), [leader](<https://devfeed.tech/tags/leader.md>)

### AI overview

Heave-Ho coordinates humanoid teams for long-horizon loco-manipulation without network communication. Its low-level multi-agent post-training adapts a humanoid foundation tracking model to physically coupled interactions, enabling decentralized control from local observations. A vision-language planner and speech-based leader-follower mechanism coordinate higher-level plans and synchronize execution. Trained with two identical robots, the system scales to teams of up to ten and generalizes to heterogeneous teams without retraining. On HH-Bench's 12 tasks, evaluations report that it outperforms baseline methods; real-world experiments demonstrate collaboration across challenging tasks. The supplied abstract does not specify the baselines or evaluation results in detail.

### Source excerpt

arXiv:2611.22275v1 Announce Type: cross Abstract: Collaboration promises to extend humanoid capabilities beyond what individual robots can achieve. However, realizing this potential requires coordinated whole-body interactions, coherent long-horizon planning, and precise synchronization across robots. We introduce Heave-Ho, a decentralized framework for long-horizon collaborative multi-humanoid loco-manipulation. At the low level, we develop a multi-agent post-training framework that adapts a humanoid foundation tracking model to physically coupled interactions between humanoids, enabling coordinated decentralized control from local observations alone. At the high level, we train a vision-language planner and design a speech-based leader-follower coordination mechanism to synchronize decentralized execution without network communication. Trained with two identical humanoids, our system scales to teams of up to ten robots and generalizes to heterogeneous teams without retraining. To benchmark these capabilities, we introduce HH-Bench, comprising 12 tasks involving teams of 2 to 10 humanoids, including heterogeneous teams. Systematic evaluations show that Heave-Ho consistently outperforms baseline methods, while real-world experiments demonstrate robust decentralized collaboration across challenging tasks.

## JointGV: Structure-Aware Correction of Visual World Models with Parallel Graph Dynamics for Long-Horizon Modeling

DevFeed: [JointGV: Structure-Aware Correction of Visual World Models with Parallel Graph Dynamics for Long-Horizon Modeling](<https://devfeed.tech/articles/jointgv-structure-aware-correction-of-visual-world-models-with-parallel-graph-dynamics-for-long-horizon-modeling-140048.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.22543>)

Author: Tian Lyu, Yichen Jiang, Ziyuan Cheng, Yunda Yan, Xiangyu Pan, Junjie Jiang, Jialin Yu, Boyu Ning, Pietro Li\\\`o, Fan Mo

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [World models](<https://devfeed.tech/topics/world-models.md>), [Robot Manipulation](<https://devfeed.tech/topics/robot-manipulation.md>), [contact-rich manipulation](<https://devfeed.tech/topics/contact-rich-manipulation.md>), [Graphs](<https://devfeed.tech/topics/graphs.md>)

Tags: [actions](<https://devfeed.tech/tags/actions.md>), [actor](<https://devfeed.tech/tags/actor.md>), [contact-rich-manipulation](<https://devfeed.tech/tags/contact-rich-manipulation.md>), [control](<https://devfeed.tech/tags/control.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-ro](<https://devfeed.tech/tags/cs-ro.md>), [graph](<https://devfeed.tech/tags/graph.md>), [horizon](<https://devfeed.tech/tags/horizon.md>), [structure](<https://devfeed.tech/tags/structure.md>), [world-models](<https://devfeed.tech/tags/world-models.md>)

### AI overview

JointGV is a plug-in that uses a task-conditioned scene graph to correct long-horizon drift in visual world models without sending graph-learning or correction gradients into the original visual dynamics. It rolls out graph and visual predictions in parallel, couples their predicted changes, and uses graph-motion context and predicted node positions to compute a bounded correction. On DINO-WM and DreamerV3, 32-step evaluations report 11-28% lower joint-position error. For DINO-WM spatial tokens, learned graph-visual change coupling outperforms giving every token the same mean graph-motion context, especially when distinct parts look close together. On real-robot inverted-pendulum sequences, DINO-WM's pole-angle error falls from 12.56° to 6.69°; DreamerV3 and LeWorldModel also improve. In simulated pendulum control, an actor using corrected rollouts completes 46 of 50 episodes, versus none with original rollouts. The supplied excerpt does not provide further methodological or evaluation details.

### Source excerpt

arXiv:2611.22543v1 Announce Type: cross Abstract: Visual world models predict future visual states conditioned on actions, but recursive errors in entity poses and spatial relations limit their long-horizon prediction for robot planning and control. We ask whether a separately predicted relational state can correct this drift without restructuring the visual model. This is challenging because graph nodes and visual states share neither elements nor a feature space. We propose Joint Graph--Visual Dynamics Correction (JointGV), a plug-in based on a cross-granularity dynamics hypothesis: despite their different representations, graph-node and visual-state changes describe the same physical transition. JointGV rolls out a task-conditioned scene graph alongside the visual world model, couples their predicted changes, and combines the resulting graph-motion context with graph-predicted node positions to compute a bounded residual. Graph-learning and correction gradients do not propagate into the original visual dynamics. We attach JointGV to DINO-WM (spatial tokens) and DreamerV3 (a global latent state), evaluating both on DMC Cheetah Run and Walker Walk across three seeds and extending the evaluation to contact-rich manipulation with DMC Finger Turn Hard. Across 32-step rollouts, JointGV improves structural and visual-prediction accuracy, reducing joint-position error by 11--28\%. For spatial tokens, learned graph--visual change coupling gives lower error than broadcasting the same mean graph-motion context to every token, particularly when non-adjacent parts appear close together in the image. On real-robot inverted-pendulum sequences, JointGV reduces DINO-WM's 32-step pole-angle error from $12.56^\circ$ to $6.69^\circ$ and improves DreamerV3 and LeWorldModel as well. In simulated inverted-pendulum control, an actor trained with JointGV-corrected rollouts completes 46 of 50 episodes, compared with none using the original rollouts.

## FailureGen: Scalable Robot Failure Data Generation with World Models

DevFeed: [FailureGen: Scalable Robot Failure Data Generation with World Models](<https://devfeed.tech/articles/failuregen-scalable-robot-failure-data-generation-with-world-models-140018.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.22367>)

Author: Akul Santhosh, Chaitra Sathe, Aastha Jhunjhunwala

Published: 2026-11-18T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Robot Manipulation](<https://devfeed.tech/topics/robot-manipulation.md>), [World models](<https://devfeed.tech/topics/world-models.md>), [Cosmos](<https://devfeed.tech/topics/cosmos.md>)

Tags: [annotations](<https://devfeed.tech/tags/annotations.md>), [continuous](<https://devfeed.tech/tags/continuous.md>), [cosmos](<https://devfeed.tech/tags/cosmos.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [cs-ro](<https://devfeed.tech/tags/cs-ro.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [detection](<https://devfeed.tech/tags/detection.md>), [failure](<https://devfeed.tech/tags/failure.md>), [fine-tuning](<https://devfeed.tech/tags/fine-tuning.md>), [generation](<https://devfeed.tech/tags/generation.md>), [learning](<https://devfeed.tech/tags/learning.md>), [motion](<https://devfeed.tech/tags/motion.md>), [precision](<https://devfeed.tech/tags/precision.md>), [recordings](<https://devfeed.tech/tags/recordings.md>), [robot](<https://devfeed.tech/tags/robot.md>), [spatial](<https://devfeed.tech/tags/spatial.md>), [study](<https://devfeed.tech/tags/study.md>), [synthetic](<https://devfeed.tech/tags/synthetic.md>), [tasks](<https://devfeed.tech/tags/tasks.md>), [temporal](<https://devfeed.tech/tags/temporal.md>), [video](<https://devfeed.tech/tags/video.md>), [world-models](<https://devfeed.tech/tags/world-models.md>)

### AI overview

FailureGen generates robot-failure video data by reusing motion from recorded failures in successful execution contexts. It fine-tunes an action-conditioned Cosmos world model on successful and failed runs, identifies failure-associated donor segments and compatible successful hosts, transfers donor motion (optionally with spatial offsets), then annotates and filters the generated videos for observed events and visual quality. The study tests two implementations across different tasks and robot configurations. In a preliminary visual failure-detection comparison, training on real plus generated data yields average precision of 0.51 versus 0.41 with real data alone, and detects 33 of 36 failure intervals versus 28 of 36. The abstract presents this as a way to expand failure data for downstream study and learning.

### Source excerpt

arXiv:2611.22367v1 Announce Type: cross Abstract: Robot failures are difficult to collect at scale, limiting the range of failure scenarios represented in robot learning datasets. We present FailureGen, a method for generating synthetic videos with temporal event annotations by reusing action trajectories from recorded failures. The method comprises four stages: (1) fine-tuning an action-conditioned Cosmos world model on successful and failed real executions; (2) annotating source recordings to identify failure-associated donor segments and compatible successful hosts; (3) transferring donor motion into host contexts, with optional spatial offsets, to generate continuous video rollouts; and (4) annotating and filtering the generated samples according to observed events and visual quality. We study two implementations with different tasks and robot configurations, with visual failure detection serving as one downstream application. In a preliminary comparison, a detector trained on real and generated data achieves an average precision of 0.51, compared with 0.41 using real data alone, and detects 33 of 36 failure intervals, compared with 28 of 36. By reusing observed failures in additional contexts, FailureGen expands the data available for studying robot failures and developing downstream learning methods.

## General-purpose robot maker Genesis AI reels in $500M at $3.5B valuation

DevFeed: [General-purpose robot maker Genesis AI reels in $500M at $3.5B valuation](<https://devfeed.tech/articles/general-purpose-robot-maker-genesis-ai-reels-in-500m-at-3-5b-valuation-139408.md>)

Original publisher: [Read original article](<https://siliconangle.com/2026/11/17/general-purpose-robot-maker-genesis-ai-reels-in-500m-at-3-5b-valuation/>)

Author: Maria Deutscher

Published: 2026-11-18T01:29:33Z

Content type: news

Language: en

Sources: [SiliconANGLE](<https://devfeed.tech/sources/siliconangle.md>)

Topics: [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Robot Manipulation](<https://devfeed.tech/topics/robot-manipulation.md>), [Frontier AI](<https://devfeed.tech/topics/frontier-ai.md>)

Tags: [advanced](<https://devfeed.tech/tags/advanced.md>), [advanced-micro-devices](<https://devfeed.tech/tags/advanced-micro-devices.md>), [ai](<https://devfeed.tech/tags/ai.md>), [automation](<https://devfeed.tech/tags/automation.md>), [capital](<https://devfeed.tech/tags/capital.md>), [communications](<https://devfeed.tech/tags/communications.md>), [devices](<https://devfeed.tech/tags/devices.md>), [eclipse](<https://devfeed.tech/tags/eclipse.md>), [electronics](<https://devfeed.tech/tags/electronics.md>), [emerging-tech](<https://devfeed.tech/tags/emerging-tech.md>), [eno](<https://devfeed.tech/tags/eno.md>), [ericsson](<https://devfeed.tech/tags/ericsson.md>), [funding](<https://devfeed.tech/tags/funding.md>), [funding-round](<https://devfeed.tech/tags/funding-round.md>), [general](<https://devfeed.tech/tags/general.md>), [genesis](<https://devfeed.tech/tags/genesis.md>), [genesis-ai](<https://devfeed.tech/tags/genesis-ai.md>), [google](<https://devfeed.tech/tags/google.md>), [humanoid-robot](<https://devfeed.tech/tags/humanoid-robot.md>), [inc](<https://devfeed.tech/tags/inc.md>), [invest](<https://devfeed.tech/tags/invest.md>), [maker](<https://devfeed.tech/tags/maker.md>), [news](<https://devfeed.tech/tags/news.md>), [reels](<https://devfeed.tech/tags/reels.md>), [robot](<https://devfeed.tech/tags/robot.md>), [startup](<https://devfeed.tech/tags/startup.md>), [the-latest](<https://devfeed.tech/tags/the-latest.md>), [valuation](<https://devfeed.tech/tags/valuation.md>)

### AI overview

Genesis AI raised $500 million in a Series A led by Premji Invest, Khosla Ventures and Eclipse, at a reported $3.5 billion valuation. Its Eno robot uses a height-adjustable wheeled base and human-like hands with 20 degrees of freedom. Genesis says the hand design lets it train GENE on human task demonstrations, which are easier to collect than footage of robots doing the same work. The company says GENE can handle tasks with dozens of steps; demos include piano playing, cooking and solving a Rubik's Cube. Genesis plans to use the funding to hire AI researchers and roboticists and collect more training data. These capabilities and claims are attributed to the company; the supplied text does not provide independent performance results.

### Source excerpt

Genesis AI SAS today announced that it has raised $500 million in funding from a group of prominent investors. Premji Invest, Khosla Ventures and Eclipse led the Series A round. They were joined by the venture capital arms of Nvidia Corp., Advanced Micro Devices Inc., Zoom Communications Inc., Samsung Electronics Co. and Ericsson. Former Google [...] The post General-purpose robot maker Genesis AI reels in $500M at $3.5B valuation appeared first on SiliconANGLE.

[Next page](<https://devfeed.tech/topics/robot-manipulation.md?cursor=WyIyMDI2LTExLTE4VDAxOjI5OjMzKzAwOjAwIiwgImQyYmVhNzhhLTcyMjktNDFhNS1hYjY2LTRjYWU5NTBkZmI4MyJd>)