{"ID":2868773,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2509.19372","arxiv_id":"2509.19372","title":"Representation-based Broad Hallucination Detectors Fail to Generalize Out of Distribution","abstract":"We critically assess the efficacy of the current SOTA in hallucination detection and find that its performance on the RAGTruth dataset is largely driven by a spurious correlation with data. Controlling for this effect, state-of-the-art performs no better than supervised linear probes, while requiring extensive hyperparameter tuning across datasets. Out-of-distribution generalization is currently out of reach, with all of the analyzed methods performing close to random. We propose a set of guidelines for hallucination detection and its evaluation.","short_abstract":"We critically assess the efficacy of the current SOTA in hallucination detection and find that its performance on the RAGTruth dataset is largely driven by a spurious correlation with data. Controlling for this effect, state-of-the-art performs no better than supervised linear probes, while requiring extensive hyperpar...","url_abs":"https://arxiv.org/abs/2509.19372","url_pdf":"https://arxiv.org/pdf/2509.19372v1","authors":"[\"Zuzanna Dubanowska\",\"Maciej Żelaszczyk\",\"Michał Brzozowski\",\"Paolo Mandica\",\"Michał Karpowicz\"]","published":"2025-09-19T10:54:22Z","proceeding":"cs.LG","tasks":"[\"cs.LG\",\"cs.AI\"]","methods":"[]","has_code":false}