{"ID":6536125,"CreatedAt":"2026-07-14T01:21:01.169441415Z","UpdatedAt":"2026-07-15T03:28:55.185153975Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2607.10569","arxiv_id":"2607.10569","title":"When Does Restricting a Coding Agent to execute_code Help? A Regime $\\times$ Agent-Design Ablation","abstract":"Modern coding agents expose multiple tool surfaces -- IDE primitives, bash, and Model Context Protocol (MCP) code-execution -- and the field has shipped three contradictory claims about which one matters. We run the missing crossed comparison: an integrity-clean three-arm ablation (baseline / bash_only / code_only) on synthetic computation tasks and SWE-bench Mini modification tasks, holding model, harness, and prompts fixed, with two agents (Claude Code, OpenAI Codex CLI) so the comparison spans both regime and agent-design axes. Across the four resulting (regime, agent) cells, restricting the agent to a single execute_code MCP tool is cheaper than -- or statistically tied with -- its cheapest tool-rich rival in three cells (significantly on Artifact/Claude and SWE-bench/Codex; directionally on Artifact/Codex), with pass rates statistically tied within each cell. The lone exception is SWE-bench/Claude, where code_only is directionally costlier (+14.4%, not significant); a conditional-cost analysis localizes that gap to failure-cost on doomed-run trajectories, not a per-edit tax on successful runs. Two implications: the cheapest tool surface is jointly determined by task regime and agent design rather than by either axis alone, and the headline cost signal lives in cache-adjusted cost -- not pass rate, which is invariant across surfaces at the model sizes we evaluate. The benchmark harness, task suite, and analysis code are available at https://github.com/hyang0129/onlycodes.","short_abstract":"Modern coding agents expose multiple tool surfaces -- IDE primitives, bash, and Model Context Protocol (MCP) code-execution -- and the field has shipped three contradictory claims about which one matters. We run the missing crossed comparison: an integrity-clean three-arm ablation (baseline / bash_only / code_only) on...","url_abs":"https://arxiv.org/abs/2607.10569","url_pdf":"https://arxiv.org/pdf/2607.10569v1","authors":"[\"Hong Yang\",\"Qi Yu\",\"Travis Desell\"]","published":"2026-07-12T04:52:08Z","proceeding":"cs.SE","tasks":"[\"cs.SE\",\"cs.AI\",\"cs.LG\"]","methods":"[]","has_code":false,"code_links":[{"ID":614135,"CreatedAt":"2026-07-14T01:21:01.169441415Z","UpdatedAt":"2026-07-14T01:21:01.169441415Z","DeletedAt":null,"paper_id":6536125,"paper_url":"https://arxiv.org/abs/2607.10569","paper_title":"When Does Restricting a Coding Agent to execute_code Help? A Regime $\\times$ Agent-Design Ablation","repo_url":"https://github.com/hyang0129/onlycodes","is_official":false,"mentioned_in_paper":false,"mentioned_in_github":true,"github_stars":0}]}