{"ID":2896336,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2507.08203","arxiv_id":"2507.08203","title":"TruthTorchLM: A Comprehensive Library for Predicting Truthfulness in LLM Outputs","abstract":"Generative Large Language Models (LLMs)inevitably produce untruthful responses. Accurately predicting the truthfulness of these outputs is critical, especially in high-stakes settings. To accelerate research in this domain and make truthfulness prediction methods more accessible, we introduce TruthTorchLM an open-source, comprehensive Python library featuring over 30 truthfulness prediction methods, which we refer to as Truth Methods. Unlike existing toolkits such as Guardrails, which focus solely on document-grounded verification, or LM-Polygraph, which is limited to uncertainty-based methods, TruthTorchLM offers a broad and extensible collection of techniques. These methods span diverse tradeoffs in computational cost, access level (e.g., black-box vs white-box), grounding document requirements, and supervision type (self-supervised or supervised). TruthTorchLM is seamlessly compatible with both HuggingFace and LiteLLM, enabling support for locally hosted and API-based models. It also provides a unified interface for generation, evaluation, calibration, and long-form truthfulness prediction, along with a flexible framework for extending the library with new methods. We conduct an evaluation of representative truth methods on three datasets, TriviaQA, GSM8K, and FactScore-Bio. The code is available at https://github.com/Ybakman/TruthTorchLM","short_abstract":"Generative Large Language Models (LLMs)inevitably produce untruthful responses. Accurately predicting the truthfulness of these outputs is critical, especially in high-stakes settings. To accelerate research in this domain and make truthfulness prediction methods more accessible, we introduce TruthTorchLM an open-sourc...","url_abs":"https://arxiv.org/abs/2507.08203","url_pdf":"https://arxiv.org/pdf/2507.08203v1","authors":"[\"Duygu Nur Yaldiz\",\"Yavuz Faruk Bakman\",\"Sungmin Kang\",\"Alperen Öziş\",\"Hayrettin Eren Yildiz\",\"Mitash Ashish Shah\",\"Zhiqi Huang\",\"Anoop Kumar\",\"Alfy Samuel\",\"Daben Liu\",\"Sai Praneeth Karimireddy\",\"Salman Avestimehr\"]","published":"2025-07-10T22:23:51Z","proceeding":"cs.CL","tasks":"[\"cs.CL\"]","methods":"[\"Large Language Model\",\"Language Model\"]","has_code":false,"code_links":[{"ID":612267,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_id":2896336,"paper_url":"https://arxiv.org/abs/2507.08203","paper_title":"TruthTorchLM: A Comprehensive Library for Predicting Truthfulness in LLM Outputs","repo_url":"https://github.com/Ybakman/TruthTorchLM","is_official":false,"mentioned_in_paper":false,"mentioned_in_github":true,"github_stars":0}]}