{"ID":5675166,"CreatedAt":"2026-07-03T01:40:09.565152011Z","UpdatedAt":"2026-07-10T01:11:38.759438437Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2607.01763","arxiv_id":"2607.01763","title":"Denser $\\neq$ Better: Limits of On-Policy Self-Distillation for Continual Post-Training","abstract":"Continual post-training enables foundation models to acquire new knowledge while preserving existing capabilities. Recent work suggests that on-policy learning can mitigate forgetting, with on-policy self-distillation emerging as a particularly attractive approach. In this work, we revisit this optimistic view through self-distillation policy optimization (SDPO). Our experiments show that SDPO can accelerate in-domain specialization when teacher signals are stable and well aligned, but it struggles to generalize to out-of-distribution scenarios. In continual post-training, SDPO exhibits stronger forgetting and can even collapse, whereas on-policy reinforcement learning methods such as GRPO adapt more conservatively and better preserve prior capabilities. Further analyses reveal that denser self-distillation induces larger drift in both parameter space and response space, and can amplify high-frequency formatting artifacts through a self-reinforcing teacher--student loop. These findings suggest that on-policy data alone is insufficient for continual learning. Dense self-distillation can accelerate specialization when teacher targets are stable and token-level supervision is reliable, but it should not be treated as a default stabilizer for continual post-training. Our code is available at https://github.com/Moenupa/SDPO-CL.","short_abstract":"Continual post-training enables foundation models to acquire new knowledge while preserving existing capabilities. Recent work suggests that on-policy learning can mitigate forgetting, with on-policy self-distillation emerging as a particularly attractive approach. In this work, we revisit this optimistic view through...","url_abs":"https://arxiv.org/abs/2607.01763","url_pdf":"https://arxiv.org/pdf/2607.01763v1","authors":"[\"Meng Wang\",\"Haohan Zhao\",\"Wenzhuo Liu\",\"Lu Yang\",\"Geng Liu\",\"Haiyang Guo\",\"Guo-Sen Xie\",\"Gaofeng Meng\",\"Hongbin Liu\",\"Fei Zhu\"]","published":"2026-07-02T06:24:30Z","proceeding":"cs.LG","tasks":"[\"cs.LG\",\"cs.CL\"]","methods":"[\"Reinforcement Learning\"]","has_code":false,"code_links":[{"ID":613881,"CreatedAt":"2026-07-03T01:40:09.565152011Z","UpdatedAt":"2026-07-03T01:40:09.565152011Z","DeletedAt":null,"paper_id":5675166,"paper_url":"https://arxiv.org/abs/2607.01763","paper_title":"Denser $\\neq$ Better: Limits of On-Policy Self-Distillation for Continual Post-Training","repo_url":"https://github.com/Moenupa/SDPO-CL","is_official":false,"mentioned_in_paper":false,"mentioned_in_github":true,"github_stars":0}]}