{"ID":2836374,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2511.21322","arxiv_id":"2511.21322","title":"TALES: A Taxonomy and Analysis of Cultural Representations in LLM-generated Stories","abstract":"Millions of users across the globe turn to AI chatbots for their creative needs, inviting widespread interest in understanding how they represent diverse cultures. However, evaluating cultural representations in open-ended tasks remains challenging and underexplored. In this work, we present TALES, an evaluation of cultural misrepresentations in LLM-generated stories for diverse Indian cultural identities. First, we develop TALES-Tax, a taxonomy of cultural misrepresentations by collating insights from participants with lived experiences in India through focus groups (N=9) and individual surveys (N=15). Using TALES-Tax, we evaluate 6 models through a large-scale annotation study spanning 2925 annotations from 108 annotators with lived experience and native language proficiency from across 71 regions in India and 14 languages. Concerningly, we find that 88% of the generated stories contain misrepresentations, and such errors are more prevalent in mid- and low-resourced languages and stories based in peri-urban regions in India. We also transform the annotations into TALES-QA, a standalone question bank to evaluate the cultural knowledge of models.","short_abstract":"Millions of users across the globe turn to AI chatbots for their creative needs, inviting widespread interest in understanding how they represent diverse cultures. However, evaluating cultural representations in open-ended tasks remains challenging and underexplored. In this work, we present TALES, an evaluation of cul...","url_abs":"https://arxiv.org/abs/2511.21322","url_pdf":"https://arxiv.org/pdf/2511.21322v2","authors":"[\"Kirti Bhagat\",\"Shaily Bhatt\",\"Athul Velagapudi\",\"Aditya Vashistha\",\"Shachi Dave\",\"Danish Pruthi\"]","published":"2025-11-26T12:07:32Z","proceeding":"cs.HC","tasks":"[\"cs.HC\",\"cs.AI\",\"cs.CL\",\"cs.CY\"]","methods":"[\"Large Language Model\"]","has_code":false}