{"ID":5438759,"CreatedAt":"2026-07-01T01:17:58.482524686Z","UpdatedAt":"2026-07-03T09:10:46.706950747Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2606.31392","arxiv_id":"2606.31392","title":"ReGRPO: Reflection-Augmented Policy Optimization for Tool-Using Agents","abstract":"Tool-augmented vision-language models (VLMs) can solve multimodal, multi-step tasks by calling external tools, yet they remain fragile in practice. Existing works have two common gaps. Supervised fine-tuning (SFT) is built mostly on successful trajectories and offers little signal for recovery after tool failures, while sparse trajectory-level RL rewards provide limited guidance on which step failed and how to repair it. We introduce ReGRPO (Reflection-augmented Group Relative Policy Optimization), a framework that learns reflection-guided correction in tool-using agents. ReGRPO starts with a structured reflective data engine: we execute near-miss actions to collect grounded failure observations, then build Reflection-of-Thought triplets (ErrorType, Evidence, FixPlan) paired with corrected actions for warm-start SFT. We then optimize reflection tokens and corrective actions jointly within local trajectories using group-relative advantages, and include a reflection-cost term to reduce unnecessary reflection. Experiments on GTA and GAIA show that, under the same backbone and tool suite, ReGRPO consistently outperforms strong open-source baselines and achieves the best results among the compared open-source controllers. Code and RoT data are available at https://github.com/showlab/ReGRPO.","short_abstract":"Tool-augmented vision-language models (VLMs) can solve multimodal, multi-step tasks by calling external tools, yet they remain fragile in practice. Existing works have two common gaps. Supervised fine-tuning (SFT) is built mostly on successful trajectories and offers little signal for recovery after tool failures, whil...","url_abs":"https://arxiv.org/abs/2606.31392","url_pdf":"https://arxiv.org/pdf/2606.31392v1","authors":"[\"Binjie Zhang\",\"Mike Zheng Shou\"]","published":"2026-06-30T09:19:38Z","proceeding":"cs.AI","tasks":"[\"cs.AI\"]","methods":"[\"Language Model\"]","has_code":false,"code_links":[{"ID":613778,"CreatedAt":"2026-07-01T01:17:58.482524686Z","UpdatedAt":"2026-07-01T01:17:58.482524686Z","DeletedAt":null,"paper_id":5438759,"paper_url":"https://arxiv.org/abs/2606.31392","paper_title":"ReGRPO: Reflection-Augmented Policy Optimization for Tool-Using Agents","repo_url":"https://github.com/showlab/ReGRPO","is_official":false,"mentioned_in_paper":false,"mentioned_in_github":true,"github_stars":0}]}