{"ID":2885329,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2508.05535","arxiv_id":"2508.05535","title":"Mixed-Initiative Dialog for Human-Robot Collaborative Manipulation","abstract":"Effective robotic systems for long-horizon human-robot collaboration must adapt to a wide range of human partners, whose physical behavior, willingness to assist, and understanding of the robot's capabilities may change over time. This demands a tightly coupled communication loop that grants both agents the flexibility to propose, accept, or decline requests as they coordinate toward completing the task effectively. We apply a Mixed-Initiative dialog paradigm to Collaborative human-roBot teaming and propose MICoBot, a system that handles the common scenario where both agents, using natural language, take initiative in formulating, accepting, or rejecting proposals on who can best complete different steps of a task. To handle diverse, task-directed dialog, and find successful collaborative strategies that minimize human effort, MICoBot makes decisions at three levels: (1) a meta-planner considers human dialog to formulate and code a high-level collaboration strategy, (2) a planner optimally allocates the remaining steps to either agent based on the robot's capabilities (measured by a simulation-pretrained affordance model) and the human's estimated availability to help, and (3) an action executor decides the low-level actions to perform or words to say to the human. In physical robot trials with 18 unique human participants, MICoBot significantly improves task success and user experience over a pure LLM baseline and standard agent allocation models. See additional videos and materials at https://robin-lab.cs.utexas.edu/MicoBot/.","short_abstract":"Effective robotic systems for long-horizon human-robot collaboration must adapt to a wide range of human partners, whose physical behavior, willingness to assist, and understanding of the robot's capabilities may change over time. This demands a tightly coupled communication loop that grants both agents the flexibility...","url_abs":"https://arxiv.org/abs/2508.05535","url_pdf":"https://arxiv.org/pdf/2508.05535v2","authors":"[\"Albert Yu\",\"Chengshu Li\",\"Luca Macesanu\",\"Arnav Balaji\",\"Ruchira Ray\",\"Raymond Mooney\",\"Roberto Martín-Martín\"]","published":"2025-08-07T16:09:12Z","proceeding":"cs.RO","tasks":"[\"cs.RO\",\"cs.CL\",\"cs.HC\",\"cs.LG\",\"cs.MA\"]","methods":"[\"Large Language Model\"]","project_urls":"[\"https://robin-lab.cs.utexas.edu/MicoBot/\"]","has_code":false,"code_links":[{"ID":611182,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_id":2885329,"paper_url":"https://arxiv.org/abs/2508.05535","paper_title":"Mixed-Initiative Dialog for Human-Robot Collaborative Manipulation","repo_url":"https://github.com/eliahuhorwitz/Academic-project-page-template","is_official":false,"mentioned_in_paper":false,"mentioned_in_github":true,"github_stars":0}]}