{"ID":6024157,"CreatedAt":"2026-07-08T01:00:23.257252134Z","UpdatedAt":"2026-07-09T22:11:23.825470046Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2607.05666","arxiv_id":"2607.05666","title":"What Do AI Agents Actually Change? An Empirical Taxonomy of Mutation Patterns in Performance-Improving Pull Requests","abstract":"AI coding agents are black boxes: we cannot inspect how they generate code, but we can inspect what they change. This distinction matters for search-based software engineering (SBSE), where techniques such as genetic improvement (in the performance-optimisation application we study) depend on mutation operators that reflect how code is actually transformed. Fewer than 1% of the 33,596 agent PRs in AIDev-pop target performance, making each case a rare window into otherwise opaque agent behaviour. We classify 1,254 performance-relevant diff hunks from 216 of these PRs, spanning five agent systems, against the 18-category syntactic mutation taxonomy of Even-Mendoza et al. (2025) using a dual-LLM intersection pipeline. Three categories dominate: name modification (37.0%), object creation (26.4%), and type change (22.7%), a profile markedly different from prior GI corpora where no change accounted for 84%. Each agent's deployed system commits to a distinctive mutation vocabulary, and each performance strategy activates a largely disjoint category subset. Agent identity and target strategy are therefore informative priors that narrow the effective SBSE operator space. Replication package: https://github.com/5uper6rain/ssbse-challenge-2026","short_abstract":"AI coding agents are black boxes: we cannot inspect how they generate code, but we can inspect what they change. This distinction matters for search-based software engineering (SBSE), where techniques such as genetic improvement (in the performance-optimisation application we study) depend on mutation operators that re...","url_abs":"https://arxiv.org/abs/2607.05666","url_pdf":"https://arxiv.org/pdf/2607.05666v1","authors":"[\"Illia Dovhoshliubnyi\",\"Nima Soroush\",\"Ashkan Sami\",\"Alexander Brownlee\"]","published":"2026-07-06T22:15:54Z","proceeding":"cs.SE","tasks":"[\"cs.SE\",\"cs.AI\"]","methods":"[\"Large Language Model\"]","has_code":false,"code_links":[{"ID":614035,"CreatedAt":"2026-07-08T01:00:23.257252134Z","UpdatedAt":"2026-07-08T01:00:23.257252134Z","DeletedAt":null,"paper_id":6024157,"paper_url":"https://arxiv.org/abs/2607.05666","paper_title":"What Do AI Agents Actually Change? An Empirical Taxonomy of Mutation Patterns in Performance-Improving Pull Requests","repo_url":"https://github.com/5uper6rain/ssbse-challenge-2026","is_official":false,"mentioned_in_paper":false,"mentioned_in_github":true,"github_stars":0}]}