{"ID":6267799,"CreatedAt":"2026-07-10T01:11:38.759438437Z","UpdatedAt":"2026-07-11T23:41:59.561444315Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2607.07993","arxiv_id":"2607.07993","title":"Hallucination Self-Play: Bootstrapping Reinforced Detector via Evolved Generator","abstract":"Identifying faithfulness hallucinations in LLM-generated outputs remains challenging due to the scarcity of high-quality annotated data. Recent work relies on advanced LLMs to synthesize training data, including rationales, labels, and hallucinated claims. However, these methods treat the generator as a static component, limiting iterative improvement of the detector. To address this limitation, we introduce Hallucination Self-Play (HSP), a novel framework that enables the detector to bootstrap with an evolved generator. HSP involves two roles initialized from the same base model, a detector that assesses the faithfulness of model outputs, and a generator that produces increasingly hard-to-detect hallucinated responses. Specifically, the detector is first fine-tuned on human-labeled data and then employed as a reward model to train the generator via reinforcement learning from AI feedback (RLAIF). In turn, the evolved generator synthesizes hallucination data to further optimize the detector through rule-based reinforcement learning. Experiments on RAGTruth benchmark and two model families demonstrate that the proposed framework can progressively enhance a small LLM to match or even outperform advanced LLMs without external supervision. Our code is available at https://anonymous.4open.science/r/Hallucination-Self-Play-50B5 .","short_abstract":"Identifying faithfulness hallucinations in LLM-generated outputs remains challenging due to the scarcity of high-quality annotated data. Recent work relies on advanced LLMs to synthesize training data, including rationales, labels, and hallucinated claims. However, these methods treat the generator as a static componen...","url_abs":"https://arxiv.org/abs/2607.07993","url_pdf":"https://arxiv.org/pdf/2607.07993v1","authors":"[\"Shiping Yang\",\"Shining Liang\",\"Weihao Liu\",\"Wenbiao Ding\",\"Linjun Shou\",\"Lu Cheng\",\"Angel X. Chang\"]","published":"2026-07-08T23:54:36Z","proceeding":"cs.CL","tasks":"[\"cs.CL\",\"cs.LG\"]","methods":"[\"Reinforcement Learning\",\"Large Language Model\"]","project_urls":"[\"https://anonymous.4open.science/r/Hallucination-Self-Play-50B5\"]","has_code":false}