{"ID":2889947,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2507.20300","arxiv_id":"2507.20300","title":"Talking-to-Build: How LLM-Assisted Interface Shapes Player Performance and Experience in Minecraft","abstract":"With large language models (LLMs) on the rise, in-game interactions are shifting from rigid commands to natural conversations. However, the impacts of LLMs on player performance and game experience remain underexplored. This work explores LLM's role as a co-builder during gameplay, examining its impact on task performance, usability, and player experience. Using Minecraft as a sandbox, we present an LLM-assisted interface that engages players through natural language, aiming to facilitate creativity and simplify complex gaming commands. We conducted a mixed-methods study with 30 participants, comparing LLM-assisted and command-based interfaces across simple and complex game tasks. Quantitative and qualitative analyses reveal that the LLM-assisted interface significantly improves player performance, engagement, and overall game experience. Additionally, task complexity has a notable effect on player performance and experience across both interfaces. Our findings highlight the potential of LLM-assisted interfaces to revolutionize virtual experiences, emphasizing the importance of balancing intuitiveness with predictability, transparency, and user agency in AI-driven, multimodal gaming environments.","short_abstract":"With large language models (LLMs) on the rise, in-game interactions are shifting from rigid commands to natural conversations. However, the impacts of LLMs on player performance and game experience remain underexplored. This work explores LLM's role as a co-builder during gameplay, examining its impact on task performa...","url_abs":"https://arxiv.org/abs/2507.20300","url_pdf":"https://arxiv.org/pdf/2507.20300v2","authors":"[\"Xin Sun\",\"Lei Wang\",\"Yue Li\",\"Jie Li\",\"Massimo Poesio\",\"Julian Frommel\",\"Koen Hinriks\",\"Jiahuan Pei\"]","published":"2025-07-27T14:34:29Z","proceeding":"cs.HC","tasks":"[\"cs.HC\",\"cs.MM\"]","methods":"[\"Large Language Model\",\"Language Model\"]","has_code":false}