{"ID":2825091,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2512.22396","arxiv_id":"2512.22396","title":"HalluMat: Detecting Hallucinations in LLM-Generated Materials Science Content Through Multi-Stage Verification","abstract":"Artificial Intelligence (AI), particularly Large Language Models (LLMs), is transforming scientific discovery, enabling rapid knowledge generation and hypothesis formulation. However, a critical challenge is hallucination, where LLMs generate factually incorrect or misleading information, compromising research integrity. To address this, we introduce HalluMatData, a benchmark dataset for evaluating hallucination detection methods, factual consistency, and response robustness in AI-generated materials science content. Alongside this, we propose HalluMatDetector, a multi-stage hallucination detection framework that integrates intrinsic verification, multi-source retrieval, contradiction graph analysis, and metric-based assessment to detect and mitigate LLM hallucinations. Our findings reveal that hallucination levels vary significantly across materials science subdomains, with high-entropy queries exhibiting greater factual inconsistencies. By utilizing HalluMatDetector verification pipeline, we reduce hallucination rates by 30% compared to standard LLM outputs. Furthermore, we introduce the Paraphrased Hallucination Consistency Score (PHCS) to quantify inconsistencies in LLM responses across semantically equivalent queries, offering deeper insights into model reliability.","short_abstract":"Artificial Intelligence (AI), particularly Large Language Models (LLMs), is transforming scientific discovery, enabling rapid knowledge generation and hypothesis formulation. However, a critical challenge is hallucination, where LLMs generate factually incorrect or misleading information, compromising research integrit...","url_abs":"https://arxiv.org/abs/2512.22396","url_pdf":"https://arxiv.org/pdf/2512.22396v1","authors":"[\"Bhanu Prakash Vangala\",\"Sajid Mahmud\",\"Pawan Neupane\",\"Joel Selvaraj\",\"Jianlin Cheng\"]","published":"2025-12-26T22:16:12Z","proceeding":"cs.AI","tasks":"[\"cs.AI\",\"cond-mat.mtrl-sci\",\"cs.IR\"]","methods":"[\"Large Language Model\",\"Language Model\"]","has_code":false}