{"ID":2862907,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2509.26476","arxiv_id":"2509.26476","title":"Regression Language Models for Code","abstract":"We study code-to-metric regression: predicting numeric outcomes of code executions, a challenging task due to the open-ended nature of programming languages. While prior methods have resorted to heavy and domain-specific feature engineering, we show that a single unified Regression Language Model (RLM) using a frozen LLM encoder can simultaneously predict directly from text, (i) the memory footprint of code across multiple high-level languages such as Python and C++, (ii) the latency of Triton GPU kernels, and (iii) the accuracy and speed of trained neural networks represented in ONNX. In particular, a relatively small 300M parameter RLM based on T5Gemma, obtains $\u003e$0.9 Spearman-rank on competitive programming submissions from APPS, and a single unified model achieves $\u003e$0.5 average Spearman-rank across 17 separate languages from CodeNet. Furthermore, the RLM can obtain the highest average Kendall-Tau of 0.46 on five classic NAS design spaces previously dominated by graph neural networks, and simultaneously predict architecture latencies on numerous hardware platforms.","short_abstract":"We study code-to-metric regression: predicting numeric outcomes of code executions, a challenging task due to the open-ended nature of programming languages. While prior methods have resorted to heavy and domain-specific feature engineering, we show that a single unified Regression Language Model (RLM) using a frozen L...","url_abs":"https://arxiv.org/abs/2509.26476","url_pdf":"https://arxiv.org/pdf/2509.26476v2","authors":"[\"Yash Akhauri\",\"Xingyou Song\",\"Arissa Wongpanich\",\"Bryan Lewandowski\",\"Mohamed S. Abdelfattah\"]","published":"2025-09-30T16:25:23Z","proceeding":"cs.CL","tasks":"[\"cs.CL\",\"cs.AI\",\"cs.LG\",\"cs.PF\",\"cs.SE\"]","methods":"[\"Graph Neural Network\",\"Large Language Model\",\"Language Model\"]","has_code":false}