{"ID":2866460,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2510.01247","arxiv_id":"2510.01247","title":"Let's Play Across Cultures: A Large Multilingual, Multicultural Benchmark for Assessing Language Models' Understanding of Sports","abstract":"Language Models (LMs) are primarily evaluated on globally popular sports, often overlooking regional and indigenous sporting traditions. To address this gap, we introduce \\textbf{\\textit{CultSportQA}}, a benchmark designed to assess LMs' understanding of traditional sports across 60 countries and 6 continents, encompassing four distinct cultural categories. The dataset features 33,000 multiple-choice questions (MCQs) across text and image modalities, each of which is categorized into three key types: history-based, rule-based, and scenario-based. To evaluate model performance, we employ zero-shot, few-shot, and chain-of-thought (CoT) prompting across a diverse set of Large Language Models (LLMs), Small Language Models (SLMs), and Multimodal Large Language Models (MLMs). By providing a comprehensive multilingual and multicultural sports benchmark, \\textbf{\\textit{CultSportQA}} establishes a new standard for assessing AI's ability to understand and reason about traditional sports.","short_abstract":"Language Models (LMs) are primarily evaluated on globally popular sports, often overlooking regional and indigenous sporting traditions. To address this gap, we introduce \\textbf{\\textit{CultSportQA}}, a benchmark designed to assess LMs' understanding of traditional sports across 60 countries and 6 continents, encompas...","url_abs":"https://arxiv.org/abs/2510.01247","url_pdf":"https://arxiv.org/pdf/2510.01247v1","authors":"[\"Punit Kumar Singh\",\"Nishant Kumar\",\"Akash Ghosh\",\"Kunal Pasad\",\"Khushi Soni\",\"Manisha Jaishwal\",\"Sriparna Saha\",\"Syukron Abu Ishaq Alfarozi\",\"Asres Temam Abagissa\",\"Kitsuchart Pasupa\",\"Haiqin Yang\",\"Jose G Moreno\"]","published":"2025-09-24T09:06:36Z","proceeding":"cs.CL","tasks":"[\"cs.CL\",\"cs.AI\"]","methods":"[\"Large Language Model\",\"Language Model\"]","has_code":false}