{"ID":2834480,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2512.01603","arxiv_id":"2512.01603","title":"MAC-SLU: Multi-Intent Automotive Cabin Spoken Language Understanding Benchmark","abstract":"Spoken Language Understanding (SLU), which aims to extract user semantics to execute downstream tasks, is a crucial component of task-oriented dialog systems. Existing SLU datasets generally lack sufficient diversity and complexity, and there is an absence of a unified benchmark for the latest Large Language Models (LLMs) and Large Audio Language Models (LALMs). This work introduces MAC-SLU, a novel Multi-Intent Automotive Cabin Spoken Language Understanding Dataset, which increases the difficulty of the SLU task by incorporating authentic and complex multi-intent data. Based on MAC-SLU, we conducted a comprehensive benchmark of leading open-source LLMs and LALMs, covering methods like in-context learning, supervised fine-tuning (SFT), and end-to-end (E2E) and pipeline paradigms. Our experiments show that while LLMs and LALMs have the potential to complete SLU tasks through in-context learning, their performance still lags significantly behind SFT. Meanwhile, E2E LALMs demonstrate performance comparable to pipeline approaches and effectively avoid error propagation from speech recognition. Code\\footnote{https://github.com/Gatsby-web/MAC\\_SLU} and datasets\\footnote{huggingface.co/datasets/Gatsby1984/MAC\\_SLU} are released publicly.","short_abstract":"Spoken Language Understanding (SLU), which aims to extract user semantics to execute downstream tasks, is a crucial component of task-oriented dialog systems. Existing SLU datasets generally lack sufficient diversity and complexity, and there is an absence of a unified benchmark for the latest Large Language Models (LL...","url_abs":"https://arxiv.org/abs/2512.01603","url_pdf":"https://arxiv.org/pdf/2512.01603v1","authors":"[\"Yuezhang Peng\",\"Chonghao Cai\",\"Ziang Liu\",\"Shuai Fan\",\"Sheng Jiang\",\"Hua Xu\",\"Yuxin Liu\",\"Qiguang Chen\",\"Kele Xu\",\"Yao Li\",\"Sheng Wang\",\"Libo Qin\",\"Xie Chen\"]","published":"2025-12-01T12:23:19Z","proceeding":"cs.CL","tasks":"[\"cs.CL\",\"cs.MM\"]","methods":"[\"Large Language Model\",\"Language Model\"]","has_code":false,"code_links":[{"ID":606419,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_id":2834480,"paper_url":"https://arxiv.org/abs/2512.01603","paper_title":"MAC-SLU: Multi-Intent Automotive Cabin Spoken Language Understanding Benchmark","repo_url":"https://github.com/Gatsby-web/MAC","is_official":false,"mentioned_in_paper":false,"mentioned_in_github":true,"github_stars":0}]}