{"ID":2872718,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2509.08803","arxiv_id":"2509.08803","title":"Scaling Truth: The Confidence Paradox in AI Fact-Checking","abstract":"The rise of misinformation underscores the need for scalable and reliable fact-checking solutions. Large language models (LLMs) hold promise in automating fact verification, yet their effectiveness across global contexts remains uncertain. We systematically evaluate nine established LLMs across multiple categories (open/closed-source, multiple sizes, diverse architectures, reasoning-based) using 5,000 claims previously assessed by 174 professional fact-checking organizations across 47 languages. Our methodology tests model generalizability on claims postdating training cutoffs and four prompting strategies mirroring both citizen and professional fact-checker interactions, with over 240,000 human annotations as ground truth. Findings reveal a concerning pattern resembling the Dunning-Kruger effect: smaller, accessible models show high confidence despite lower accuracy, while larger models demonstrate higher accuracy but lower confidence. This risks systemic bias in information verification, as resource-constrained organizations typically use smaller models. Performance gaps are most pronounced for non-English languages and claims originating from the Global South, threatening to widen existing information inequalities. These results establish a multilingual benchmark for future research and provide an evidence base for policy aimed at ensuring equitable access to trustworthy, AI-assisted fact-checking.","short_abstract":"The rise of misinformation underscores the need for scalable and reliable fact-checking solutions. Large language models (LLMs) hold promise in automating fact verification, yet their effectiveness across global contexts remains uncertain. We systematically evaluate nine established LLMs across multiple categories (ope...","url_abs":"https://arxiv.org/abs/2509.08803","url_pdf":"https://arxiv.org/pdf/2509.08803v1","authors":"[\"Ihsan A. Qazi\",\"Zohaib Khan\",\"Abdullah Ghani\",\"Agha A. Raza\",\"Zafar A. Qazi\",\"Wassay Sajjad\",\"Ayesha Ali\",\"Asher Javaid\",\"Muhammad Abdullah Sohail\",\"Abdul H. Azeemi\"]","published":"2025-09-10T17:36:25Z","proceeding":"cs.SI","tasks":"[\"cs.SI\",\"cs.AI\",\"cs.CL\",\"cs.CY\"]","methods":"[\"Large Language Model\",\"Language Model\",\"Generative Adversarial Network\"]","has_code":false}