{"ID":5346713,"CreatedAt":"2026-06-30T04:09:55.830587294Z","UpdatedAt":"2026-07-07T09:57:39.419301368Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2606.30406","arxiv_id":"2606.30406","title":"MOPD: Multi-Teacher On-Policy Distillation for Capability Integration in LLM Post-Training","abstract":"Modern large language models (LLMs) rely on reinforcement learning during post-training to push specific capabilities, yet integrating multiple capabilities into one model remains hard. Existing methods, such as Off-Policy Finetune and Mix-RL, are either inefficient or lose performance. In this work, we propose Multi-teacher On-Policy Distillation (MOPD), a post-training paradigm for combining the capabilities of multiple domain RL teachers: we first run per-domain specialised RL to obtain a set of domain teachers, then distill these teachers into the student on its own rollouts. This eliminates exposure bias and provides a dense optimization signal. On Qwen3-30B-A3B, MOPD outperforms Mix-RL, Cascade RL, Off-Policy Finetune, and Param-Merge baselines, inheriting nearly all of each teacher's capability. MOPD also enables parallel, independent development of domain teachers, removing the cross-domain coupling typical of multi-domain post-training. MOPD has been deployed in the post-training of MiMo-V2-Flash, an industrial-scale frontier model, demonstrating its practical value for capability integration in frontier-scale LLMs.","short_abstract":"Modern large language models (LLMs) rely on reinforcement learning during post-training to push specific capabilities, yet integrating multiple capabilities into one model remains hard. Existing methods, such as Off-Policy Finetune and Mix-RL, are either inefficient or lose performance. In this work, we propose Multi-t...","url_abs":"https://arxiv.org/abs/2606.30406","url_pdf":"https://arxiv.org/pdf/2606.30406v1","authors":"[\"Wenhan Ma\",\"Jianyu Wei\",\"Liang Zhao\",\"Hailin Zhang\",\"Bangjun Xiao\",\"Lei Li\",\"Qibin Yang\",\"Bofei Gao\",\"Yudong Wang\",\"Rang Li\",\"Jinhao Dong\",\"Zhifang Sui\",\"Fuli Luo\"]","published":"2026-06-29T14:51:28Z","proceeding":"cs.CL","tasks":"[\"cs.CL\",\"cs.LG\"]","methods":"[\"Reinforcement Learning\",\"Large Language Model\",\"Language Model\"]","has_code":false}