{"ID":6267445,"CreatedAt":"2026-07-10T01:11:38.759438437Z","UpdatedAt":"2026-07-11T08:29:44.454854982Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2607.07744","arxiv_id":"2607.07744","title":"PERFOPT-Bench: Evaluating Coding Agents on Software Performance Optimization","abstract":"Coding-agent benchmarks have largely measured whether agents can produce functionally correct patches, but production software also demands measurable speedups on real execution targets. Performance optimization is a distinct agentic task: agents must profile executions, diagnose cross-layer bottlenecks, edit code without breaking correctness, and verify that gains are reproducible rather than measurement artifacts. We introduce PERFOPT-Bench, a benchmark for evaluating this full performance-engineering loop. Each task provides a correct but deliberately suboptimal codebase and asks the agent to improve a target performance metric; scoring requires hidden correctness tests, verified-speedup measurement, and trajectory-level audit. We evaluate 7 agent stacks with different LLMs and agent frameworks on 7 long-horizon optimization tasks. The results show that optimization performance is workload-dependent rather than determined by model identity alone: no single stack dominates, and changing the agent framework can materially change the same LLM's per-task speedup profile. We further find that raw speedup is unsafe as a benchmark score, since some large gains arise from benchmark-specific shortcut exploitation; an exploratory relay pilot suggests that restarting from an externalized optimization summary can recover additional headroom after an initial session stops. The benchmark and our evaluation are available at: https://anonymous.4open.science/r/Dataset-D3CC.","short_abstract":"Coding-agent benchmarks have largely measured whether agents can produce functionally correct patches, but production software also demands measurable speedups on real execution targets. Performance optimization is a distinct agentic task: agents must profile executions, diagnose cross-layer bottlenecks, edit code with...","url_abs":"https://arxiv.org/abs/2607.07744","url_pdf":"https://arxiv.org/pdf/2607.07744v1","authors":"[\"Yingyun Cui\",\"Yi Xie\",\"Piaohong Wang\",\"Jiawei Ma\",\"Bo Liu\",\"Liangliang Cao\"]","published":"2026-07-08T08:32:18Z","proceeding":"cs.SE","tasks":"[\"cs.SE\"]","methods":"[\"Large Language Model\",\"LoRA\"]","project_urls":"[\"https://anonymous.4open.science/r/Dataset-D3CC\"]","has_code":false}