{"ID":2893743,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2507.22069","arxiv_id":"2507.22069","title":"A Compute-Matched Re-Evaluation of TroVE on MATH","abstract":"Reusing established theorems and formulas is central to mathematical problem solving, serving as essential building blocks for tackling increasingly complex challenges. Recent work, TroVE, argues that code-generating Large Language Models (LLMs) can benefit similarly on the MATH benchmark by inducing and reusing higher-level toolboxes. By allocating computational budget across an ensemble of three modes -- directly generating code, creating tools, and reusing tools -- TroVE claims to outperform a PRIMITIVE baseline that only performs direct generation. However, recent analysis (Berlot-Attwell et al., 2024) casts doubt on these gains, noting that the tools created are often trivial or rarely reused, suggesting that improvements may stem from self-consistency or self-correction. In this work, we re-evaluate TroVE on MATH, analyze the impact of each of its modes, and show that its benefit does not come from these mechanisms, but simply from a higher computational budget spent for TroVE compared to PRIMITIVE. To this end, we also perform a small correction in the original implementation of TroVE's selection mechanism, boosting TroVE's performance on MATH by 3\\% in accuracy. After matching for compute, the benefit of TroVE reduces to a marginal improvement of 1\\%, suggesting that this toolbox approach does not provide a significant benefit on MATH.","short_abstract":"Reusing established theorems and formulas is central to mathematical problem solving, serving as essential building blocks for tackling increasingly complex challenges. Recent work, TroVE, argues that code-generating Large Language Models (LLMs) can benefit similarly on the MATH benchmark by inducing and reusing higher...","url_abs":"https://arxiv.org/abs/2507.22069","url_pdf":"https://arxiv.org/pdf/2507.22069v2","authors":"[\"Tobias Sesterhenn\",\"Ian Berlot-Attwell\",\"Janis Zenkner\",\"Christian Bartelt\"]","published":"2025-07-16T03:11:43Z","proceeding":"cs.PL","tasks":"[\"cs.PL\",\"cs.AI\"]","methods":"[\"Large Language Model\",\"Language Model\"]","has_code":false}