{"ID":5438716,"CreatedAt":"2026-07-01T01:17:58.482524686Z","UpdatedAt":"2026-07-03T07:34:59.203171219Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2606.31308","arxiv_id":"2606.31308","title":"Benchmarking Large Language Models on Floating-Point Error Classification","abstract":"This paper investigates the capability of Large Language Models (LLMs) to detect and classify floating-point errors statically in software code. We introduce InterFLOPBench, a benchmark of 90 C kernels with 1 130 test samples designed to evaluate LLMs across six categories of floating-point error: cancellation, comparison, division by zero, overflow, underflow and NaN, compared across 14 LLMs. The evaluation framework treats floating-point error detection as a multi-label classification problem and employs the F1-score metric to measure performance. Results demonstrate that latest models (Qwen 3 32b, Gemini 2.5 Flash, Phi 4 Reasoning, DeepSeek R1T2, and gpt-oss 20b and 120b) achieve a performance greater than 0.88 overall F1-score. Performance varies between error categories, between explicit operations such as division by zero (Average F1-score: 0.8479) and more subtle numerical phenomena such as underflow (Average F1-score: 0.6059) and cancellation (Average F1-score: 0.6164).","short_abstract":"This paper investigates the capability of Large Language Models (LLMs) to detect and classify floating-point errors statically in software code. We introduce InterFLOPBench, a benchmark of 90 C kernels with 1 130 test samples designed to evaluate LLMs across six categories of floating-point error: cancellation, compari...","url_abs":"https://arxiv.org/abs/2606.31308","url_pdf":"https://arxiv.org/pdf/2606.31308v1","authors":"[\"Lisa Taldir\",\"Muhammad Ahmad Saeed\",\"David Defour\",\"Pablo de Oliveira Castro\",\"Eric Petit\"]","published":"2026-06-30T08:18:45Z","proceeding":"cs.AI","tasks":"[\"cs.AI\"]","methods":"[\"Large Language Model\",\"Language Model\"]","has_code":false}