{"ID":2852885,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2510.17722","arxiv_id":"2510.17722","title":"MT-Video-Bench: A Holistic Video Understanding Benchmark for Evaluating Multimodal LLMs in Multi-Turn Dialogues","abstract":"The recent development of Multimodal Large Language Models (MLLMs) has significantly advanced AI's ability to understand visual modalities. However, existing evaluation benchmarks remain limited to single-turn question answering, overlooking the complexity of multi-turn dialogues in real-world scenarios. To bridge this gap, we introduce MT-Video-Bench, a holistic video understanding benchmark for evaluating MLLMs in multi-turn dialogues. Specifically, our MT-Video-Bench mainly assesses 6 core competencies that focus on perceptivity and interactivity, encompassing 1,000 meticulously curated multi-turn dialogues from diverse domains. These capabilities are rigorously aligned with real-world applications, such as interactive sports analysis and multi-turn video-based intelligent tutoring. With MT-Video-Bench, we extensively evaluate various state-of-the-art open-source and closed-source MLLMs, revealing their significant performance discrepancies and limitations in handling multi-turn video dialogues. The benchmark will be publicly available to foster future research.","short_abstract":"The recent development of Multimodal Large Language Models (MLLMs) has significantly advanced AI's ability to understand visual modalities. However, existing evaluation benchmarks remain limited to single-turn question answering, overlooking the complexity of multi-turn dialogues in real-world scenarios. To bridge this...","url_abs":"https://arxiv.org/abs/2510.17722","url_pdf":"https://arxiv.org/pdf/2510.17722v2","authors":"[\"Yaning Pan\",\"Qianqian Xie\",\"Guohui Zhang\",\"Zekun Wang\",\"Yongqian Wen\",\"Yuanxing Zhang\",\"Haoxuan Hu\",\"Zhiyu Pan\",\"Yibing Huang\",\"Zhidong Gan\",\"Yonghong Lin\",\"An Ping\",\"Shihao Li\",\"Yanghai Wang\",\"Tianhao Peng\",\"Jiaheng Liu\"]","published":"2025-10-20T16:38:40Z","proceeding":"cs.CV","tasks":"[\"cs.CV\",\"cs.AI\"]","methods":"[\"Large Language Model\",\"Language Model\"]","has_code":false}