# InfCoiL: Coordinated Planner-Controller Learning for Closed-Loop Physics-Based Human-Object Interaction

DevFeed: [InfCoiL: Coordinated Planner-Controller Learning for Closed-Loop Physics-Based Human-Object Interaction](<https://devfeed.tech/articles/infcoil-coordinated-planner-controller-learning-for-closed-loop-physics-based-human-object-interaction-125399.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.11517>)

Author: Yude Zou, Yifei Yao, Yiwei Hao, Hanqing Wang, Xing Gao

Published: 2026-11-10T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Machine Learning](<https://devfeed.tech/sources/arxiv-machine-learning.md>)

Topics: [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [contact-rich manipulation](<https://devfeed.tech/topics/contact-rich-manipulation.md>)

Tags: [animation](<https://devfeed.tech/tags/animation.md>), [autoregressive](<https://devfeed.tech/tags/autoregressive.md>), [controller](<https://devfeed.tech/tags/controller.md>), [cs-cv](<https://devfeed.tech/tags/cs-cv.md>), [cs-lg](<https://devfeed.tech/tags/cs-lg.md>), [flow](<https://devfeed.tech/tags/flow.md>), [flow-matching](<https://devfeed.tech/tags/flow-matching.md>), [framework](<https://devfeed.tech/tags/framework.md>), [generative](<https://devfeed.tech/tags/generative.md>), [humanobject-interaction](<https://devfeed.tech/tags/humanobject-interaction.md>), [interface](<https://devfeed.tech/tags/interface.md>), [learning](<https://devfeed.tech/tags/learning.md>), [online](<https://devfeed.tech/tags/online.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [physics](<https://devfeed.tech/tags/physics.md>), [planner](<https://devfeed.tech/tags/planner.md>), [reinforcement-learning](<https://devfeed.tech/tags/reinforcement-learning.md>), [semantic](<https://devfeed.tech/tags/semantic.md>), [sequence](<https://devfeed.tech/tags/sequence.md>), [sequences](<https://devfeed.tech/tags/sequences.md>), [systems](<https://devfeed.tech/tags/systems.md>), [text](<https://devfeed.tech/tags/text.md>)

## AI overview

InfCoiL trains a human-object interaction planner and physics-based controller together in a closed loop, allowing execution feedback to reshape future plans. A morphology-aware flow-matching planner and a unified multi-morphology controller share an interface and are alternately fine-tuned with reinforcement learning on the same physical rollouts. Its Semantic Flow Policy Optimization method stochasticizes selected denoising steps and uses group-relative, sequence-level rewards from complete executions. On held-out OMOMO sequences, the abstract reports higher goal-reaching success and physical tracking accuracy than open-loop hierarchical baselines.

## Source excerpt

arXiv:2611.11517v1 Announce Type: cross Abstract: Text- and goal-conditioned human-object interaction (HOI) provides an intuitive interface for specifying high-level intents in physics-based character animation. Mapping such intents to physically plausible, contact-rich behaviors naturally calls for a generative planner and a physics-based controller, yet existing planner-controller systems either execute plans open-loop or close the loop while keeping the planner fixed, so execution feedback never reshapes what is planned. We present InfCoiL, a closed-loop framework for physics-based HOI from coordinated planner-controller learning. InfCoiL couples a morphology-aware flow-matching planner with a unified multi-morphology controller through a shared morphology interface in an autoregressive plan-and-execute loop, and alternately fine-tunes both components with reinforcement learning from the same physical rollouts. To optimize the flow-matching planner online, we introduce Semantic Flow Policy Optimization (SFPO), which stochasticizes selected denoising transitions with channel-wise semantic covariance over heterogeneous HOI features and assigns group-relative, sequence-level rewards from complete closed-loop executions. On held-out OMOMO sequences, InfCoiL outperforms open-loop hierarchical baselines in goal-reaching task success and physical tracking accuracy. Project page: https://infcoil.github.io/