{"ID":5937290,"CreatedAt":"2026-07-07T03:14:33.014478982Z","UpdatedAt":"2026-07-14T02:54:43.516908796Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2607.04751","arxiv_id":"2607.04751","title":"Trust Region Policy Distillation","abstract":"Big goals are hard to achieve all at once; breaking them into small steps is wiser. We present Trust Region Policy Distillation (TOP-D), which transforms the notoriously unstable, high-variance On-Policy Distillation (OPD) into a stable training paradigm by dynamically constructing a proximal teacher. Theoretically, we establish a rigorous framework demonstrating that TOP-D inherently controls gradient variance. By providing a formal global convergence analysis alongside a monotonic improvement bound, we mathematically formalize the reliability and stability of the overall training dynamics. Empirically, TOP-D dramatically enhances training stability, sample efficiency, and final performance on mathematical reasoning tasks. More importantly, TOP-D introduces zero additional computational overhead, positioning itself as a promising alternative to the well-established OPD paradigm.","short_abstract":"Big goals are hard to achieve all at once; breaking them into small steps is wiser. We present Trust Region Policy Distillation (TOP-D), which transforms the notoriously unstable, high-variance On-Policy Distillation (OPD) into a stable training paradigm by dynamically constructing a proximal teacher. Theoretically, we...","url_abs":"https://arxiv.org/abs/2607.04751","url_pdf":"https://arxiv.org/pdf/2607.04751v1","authors":"[\"Zhengpeng Xie\",\"Li Lyna Zhang\",\"Zeke Xie\",\"Mao Yang\"]","published":"2026-07-06T07:43:36Z","proceeding":"cs.LG","tasks":"[\"cs.LG\",\"cs.AI\"]","methods":"[]","has_code":false}