{"ID":5551771,"CreatedAt":"2026-07-02T01:54:51.863792489Z","UpdatedAt":"2026-07-04T10:10:07.702510095Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2607.00724","arxiv_id":"2607.00724","title":"MSQA: A Natively Sourced Multilingual and Multicultural SimpleQA Benchmark","abstract":"Multilingual fluency often invites a stronger assumption: a model that can speak a user's language must also understand the culture encoded by that language. We call this the Illusion of Cultural Alignment. To test this assumption directly, we introduce MSQA, a benchmark of 1,064 natively sourced questions across 11 language groups, five cultural dimensions, and three difficulty tiers. Unlike translated benchmarks, MSQA targets locally grounded knowledge and reduces shortcuts from English-centric cross-lingual transfer. Evaluating 18 LLMs, we find substantial cultural degradation and a pronounced Locality Effect: cultural competence tracks pre-training exposure more closely than general reasoning ability. We further show that common inference-time remedies do not dissolve the illusion. Models remain overconfident on unfamiliar cultural questions, repeated sampling yields unstable rather than reliable correctness, and retrieval augmentation helps unevenly on long-tail facts. These findings indicate that cultural alignment cannot be inferred from multilingual ability alone and requires deeper intervention than calibration, sampling, or retrieval at inference time","short_abstract":"Multilingual fluency often invites a stronger assumption: a model that can speak a user's language must also understand the culture encoded by that language. We call this the Illusion of Cultural Alignment. To test this assumption directly, we introduce MSQA, a benchmark of 1,064 natively sourced questions across 11 la...","url_abs":"https://arxiv.org/abs/2607.00724","url_pdf":"https://arxiv.org/pdf/2607.00724v1","authors":"[\"Xianru Chen\",\"Yukai Huang\",\"Mingxiang Chen\",\"Xinping Lei\",\"Fangbing Deng\",\"Jin Chen\",\"Ge Zhang\",\"Wenhao Huang\",\"Jiaheng Liu\"]","published":"2026-07-01T10:12:03Z","proceeding":"cs.CL","tasks":"[\"cs.CL\"]","methods":"[\"Large Language Model\"]","has_code":false}