{"ID":2838943,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2511.16108","arxiv_id":"2511.16108","title":"SkyRL-Agent: Efficient RL Training for Multi-turn LLM Agent","abstract":"We introduce SkyRL-Agent, a framework for efficient, multi-turn, long-horizon agent training and evaluation. It provides efficient asynchronous dispatching, lightweight tool integration, and flexible backend interoperability, enabling seamless use with existing RL frameworks such as SkyRL-train, VeRL, and Tinker. Using SkyRL-Agent, we train SA-SWE-32B, a software engineering agent trained from Qwen3-32B (24.4% Pass@1) purely with reinforcement learning. We introduce two key components: an optimized asynchronous pipeline dispatcher that achieves a 1.55x speedup over naive asynchronous batching, and a tool-enhanced training recipe leveraging an AST-based search tool to facilitate code navigation, boost rollout Pass@K, and improve training efficiency. Together, these optimizations enable SA-SWE-32B to reach 39.4% Pass@1 on SWE-Bench Verified with more than 2x cost reduction compared to prior models reaching similar performance. Despite being trained solely on SWE tasks, SA-SWE-32B generalizes effectively to other agentic tasks, including Terminal-Bench, BrowseComp-Plus, and WebArena. We further demonstrate SkyRL-Agent's extensibility through case studies on deep research, computer use, and memory agents, each trained using a different training backend.","short_abstract":"We introduce SkyRL-Agent, a framework for efficient, multi-turn, long-horizon agent training and evaluation. It provides efficient asynchronous dispatching, lightweight tool integration, and flexible backend interoperability, enabling seamless use with existing RL frameworks such as SkyRL-train, VeRL, and Tinker. Using...","url_abs":"https://arxiv.org/abs/2511.16108","url_pdf":"https://arxiv.org/pdf/2511.16108v1","authors":"[\"Shiyi Cao\",\"Dacheng Li\",\"Fangzhou Zhao\",\"Shuo Yuan\",\"Sumanth R. Hegde\",\"Connor Chen\",\"Charlie Ruan\",\"Tyler Griggs\",\"Shu Liu\",\"Eric Tang\",\"Richard Liaw\",\"Philipp Moritz\",\"Matei Zaharia\",\"Joseph E. Gonzalez\",\"Ion Stoica\"]","published":"2025-11-20T07:05:19Z","proceeding":"cs.AI","tasks":"[\"cs.AI\"]","methods":"[\"Reinforcement Learning\",\"Large Language Model\"]","has_code":false}