{"ID":2866019,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2509.21143","arxiv_id":"2509.21143","title":"Automotive-ENV: Benchmarking Multimodal Agents in Vehicle Interface Systems","abstract":"Multimodal agents have demonstrated strong performance in general GUI interactions, but their application in automotive systems has been largely unexplored. In-vehicle GUIs present distinct challenges: drivers' limited attention, strict safety requirements, and complex location-based interaction patterns. To address these challenges, we introduce Automotive-ENV, the first high-fidelity benchmark and interaction environment tailored for vehicle GUIs. This platform defines 185 parameterized tasks spanning explicit control, implicit intent understanding, and safety-aware tasks, and provides structured multimodal observations with precise programmatic checks for reproducible evaluation. Building on this benchmark, we propose ASURADA, a geo-aware multimodal agent that integrates GPS-informed context to dynamically adjust actions based on location, environmental conditions, and regional driving norms. Experiments show that geo-aware information significantly improves success on safety-aware tasks, highlighting the importance of location-based context in automotive environments. We will release Automotive-ENV, complete with all tasks and benchmarking tools, to further the development of safe and adaptive in-vehicle agents.","short_abstract":"Multimodal agents have demonstrated strong performance in general GUI interactions, but their application in automotive systems has been largely unexplored. In-vehicle GUIs present distinct challenges: drivers' limited attention, strict safety requirements, and complex location-based interaction patterns. To address th...","url_abs":"https://arxiv.org/abs/2509.21143","url_pdf":"https://arxiv.org/pdf/2509.21143v2","authors":"[\"Junfeng Yan\",\"Biao Wu\",\"Meng Fang\",\"Ling Chen\"]","published":"2025-09-25T13:30:13Z","proceeding":"cs.RO","tasks":"[\"cs.RO\",\"cs.CL\"]","methods":"[]","has_code":false}