# Joi-Bench-Task: Benchmarking Task Coordination in Full-duplex Voice Agents through Interactive Simulation

DevFeed: [Joi-Bench-Task: Benchmarking Task Coordination in Full-duplex Voice Agents through Interactive Simulation](<https://devfeed.tech/articles/joi-bench-task-benchmarking-task-coordination-in-full-duplex-voice-agents-through-interactive-simulation-126232.md>)

Original publisher: [Read original article](<https://arxiv.org/abs/2611.12118>)

Author: Xiangheng He, Mingjun Zhao, Zhehao Fan, Zhengjia Li, Xiaopeng Sun, Qingqing Xiao, Haolan Chen, Yu Xu

Published: 2026-11-10T05:00:00Z

Content type: article

Language: en

Sources: [arXiv: Artificial Intelligence](<https://devfeed.tech/sources/arxiv-artificial-intelligence.md>)

Topics: [Conversational AI](<https://devfeed.tech/topics/conversational-ai.md>), [Interactive AI Systems](<https://devfeed.tech/topics/interactive-ai-systems.md>), [task management](<https://devfeed.tech/topics/task-management.md>), [Simulation](<https://devfeed.tech/topics/simulation.md>)

Tags: [adapting](<https://devfeed.tech/tags/adapting.md>), [agents](<https://devfeed.tech/tags/agents.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [benchmarking](<https://devfeed.tech/tags/benchmarking.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [challenge](<https://devfeed.tech/tags/challenge.md>), [changes](<https://devfeed.tech/tags/changes.md>), [completion](<https://devfeed.tech/tags/completion.md>), [complex](<https://devfeed.tech/tags/complex.md>), [conversations](<https://devfeed.tech/tags/conversations.md>), [cs-ai](<https://devfeed.tech/tags/cs-ai.md>), [delivery](<https://devfeed.tech/tags/delivery.md>), [dimensions](<https://devfeed.tech/tags/dimensions.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [evaluations](<https://devfeed.tech/tags/evaluations.md>), [execution](<https://devfeed.tech/tags/execution.md>), [full](<https://devfeed.tech/tags/full.md>), [interactive](<https://devfeed.tech/tags/interactive.md>), [simulation](<https://devfeed.tech/tags/simulation.md>), [voice](<https://devfeed.tech/tags/voice.md>), [voice-agents](<https://devfeed.tech/tags/voice-agents.md>)

## AI overview

Joi-Bench-Task evaluates how full-duplex voice agents coordinate ongoing tasks with conversation as users change requests or interrupt. Its interactive simulator schedules configurable interventions during tool execution and spoken result delivery, enabling reproducible tests of execution responsiveness and coordination under changes and interruptions. Across five commercial agents, the highest result completion rate after request changes was 55%; four agents had average information omission rates of at least 23% after interruption. The results point to persistent difficulty adapting to evolving conversations while tracking what remains to be completed or reported.

## Source excerpt

arXiv:2611.12118v1 Announce Type: new Abstract: Full-duplex voice agents enable natural, hands-free interaction, allowing users to refine requests, redirect ongoing work, and seek clarification while the agent is speaking. However, existing voice-agent benchmarks insufficiently isolate coordination between task execution and conversation. We introduce a benchmark that assesses this coordination by examining whether agents track what is being executed, what has been completed, what has been communicated, and what remains to be reported as the conversation evolves. We propose an interactive user simulator that introduces configurable interventions at designated timing in tool execution and spoken result delivery, supporting controllable and reproducible evaluation conditions. We evaluate five leading commercial voice agents along three dimensions: execution responsiveness, coordination under request changes, and coordination under interruptions. Agent evaluations reveal persistent coordination failures. Under request changes, the highest result completion rate is only 55%, indicating difficulty for current voice agents adapting to new requests while preserving earlier obligations. After user interruption, four of the five agents exhibit average information omission rates of at least 23%. These findings highlight task coordination as a central challenge for current voice agents in complex, evolving conversations.