{"ID":5551859,"CreatedAt":"2026-07-02T01:54:51.863792489Z","UpdatedAt":"2026-07-04T06:25:51.571775532Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2607.00531","arxiv_id":"2607.00531","title":"Active-GRPO: Adaptive Imitation and Self-Improving Reasoning for Molecular Optimization","abstract":"Scientific reasoning is an increasingly important capability of large language models, yet improving the robustness and efficiency of training such reasoning remains a key open challenge. We study this problem in instruction-based molecular optimization, where answer-only supervised fine-tuning (SFT) collapses multi-step reasoning and reinforcement learning with verifiable rewards (RLVR) suffers from sparse feedback. Reference-guided Policy Optimization mitigates both by anchoring policy updates to dataset-provided references, but its effectiveness is tightly coupled to reference quality: weak or misaligned references impose a performance ceiling. To overcome this ceiling, we propose active reasoning, a paradigm in which the policy actively decides, on a per-instance basis, when to imitate a reference and when to reinforce its own discoveries, while continuously upgrading what it imitates. We instantiate this paradigm as Active Group Relative Policy Optimization (Active-GRPO), realized through two coupled mechanisms: active imitate-reinforce and active referencing. The former performs imitation learning when the reference still outperforms the policy's own candidates, and shifts to self-improvement via reinforcement learning once the policy has generated molecules that surpass the reference. The latter continuously upgrades the reference itself by replacing it with the best policy-generated candidate discovered so far, progressively raising the imitation target and ensuring that reference guidance remains informative-rather than restrictive-throughout training. Across TOMG-Bench MOLOPT, Active-GRPO improves average SRxSim from 0.0959 for GRPO and 0.1665 for RePO to 0.1773 under matched three-seed evaluation, with statistically significant gains on LogP, MR, and QED.","short_abstract":"Scientific reasoning is an increasingly important capability of large language models, yet improving the robustness and efficiency of training such reasoning remains a key open challenge. We study this problem in instruction-based molecular optimization, where answer-only supervised fine-tuning (SFT) collapses multi-st...","url_abs":"https://arxiv.org/abs/2607.00531","url_pdf":"https://arxiv.org/pdf/2607.00531v1","authors":"[\"Xuefeng Liu\",\"Mingxuan Cao\",\"Qinan Huang\",\"Thomas Brettin\",\"Rick Stevens\",\"Le Cong\"]","published":"2026-07-01T07:22:46Z","proceeding":"cs.LG","tasks":"[\"cs.LG\",\"cs.AI\",\"q-bio.BM\",\"stat.ML\"]","methods":"[\"Reinforcement Learning\",\"Language Model\"]","has_code":false}