{"ID":5675964,"CreatedAt":"2026-07-03T01:40:09.565152011Z","UpdatedAt":"2026-07-04T19:15:18.090787218Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2607.01415","arxiv_id":"2607.01415","title":"The Rollout Infrastructure Tax in Coding-Agent Reinforcement Learning","abstract":"Coding-agent reinforcement learning treats execution infrastructure as a background implementation detail, despite relying on large numbers of interactive software rollouts. This is a missed opportunity: measuring infrastructure overhead can reveal practical efficiency gains for RL post-training, where small per-rollout savings compound at scale. We present a comparative study of four execution substrates: single containers, hosted sandboxes, Kubernetes-orchestrated containers, and cloud virtual machines. We find up to $110\\times$ variation in cold-start latency and a $1.8\\times$ spread in projected worker-hours for one million 150-step trajectories. Our results suggest that future coding-agent RL systems should optimize execution substrates as part of the training system itself, not merely as deployment plumbing.","short_abstract":"Coding-agent reinforcement learning treats execution infrastructure as a background implementation detail, despite relying on large numbers of interactive software rollouts. This is a missed opportunity: measuring infrastructure overhead can reveal practical efficiency gains for RL post-training, where small per-rollou...","url_abs":"https://arxiv.org/abs/2607.01415","url_pdf":"https://arxiv.org/pdf/2607.01415v1","authors":"[\"Daniel Thi Graviet\",\"Lovre Pesut\",\"Ivan Dagelic\",\"Vedran Jukic\",\"Ivan Burazin\"]","published":"2026-07-01T19:20:46Z","proceeding":"cs.LG","tasks":"[\"cs.LG\",\"cs.DC\"]","methods":"[\"Reinforcement Learning\"]","has_code":false}