{"ID":3084839,"CreatedAt":"2026-06-05T06:46:15.197025399Z","UpdatedAt":"2026-06-07T03:54:17.966829144Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2606.05687","arxiv_id":"2606.05687","title":"Accelerating and Scaling MPC-Guided Reinforcement Learning for Humanoid Locomotion and Manipulation","abstract":"In humanoid motion control, model predictive control (MPC) offers physically grounded prediction and constraint handling, while reinforcement learning (RL) enables robust whole-body skills through large-scale simulation. However, using MPC inside RL often requires time-consuming problem construction or excessive training overhead, making such frameworks difficult to justify in practice. This work studies efficient training-time MPC guidance for humanoid locomotion and manipulation, termed MPC-RL. We introduce a centroidal-dynamics MPC reward formulation that leverages guidance from MPC trajectories in training time. To make this practical in massively parallel RL, we develop $π^n$MPC, a parallel-in-horizon and construction-free batched GPU MPC solver that operates directly on time-varying dynamics to avoid high memory usage and pre-compilation. Through a variety of comparative studies and hardware validations, we have found that MPC-RL achieves superior performance in locomotion and manipulation skills. The code base is available at https://github.com/junhengl/mpc-rl.","short_abstract":"In humanoid motion control, model predictive control (MPC) offers physically grounded prediction and constraint handling, while reinforcement learning (RL) enables robust whole-body skills through large-scale simulation. However, using MPC inside RL often requires time-consuming problem construction or excessive traini...","url_abs":"https://arxiv.org/abs/2606.05687","url_pdf":"https://arxiv.org/pdf/2606.05687v1","authors":"[\"Junheng Li\",\"Liang Wu\",\"Sergio A. Esteban\",\"Lizhi Yang\",\"Ján Drgoňa\",\"Aaron D. Ames\"]","published":"2026-06-04T04:12:47Z","proceeding":"cs.RO","tasks":"[\"cs.RO\",\"eess.SY\"]","methods":"[\"Reinforcement Learning\"]","has_code":false,"code_links":[{"ID":612862,"CreatedAt":"2026-06-05T06:46:15.197025399Z","UpdatedAt":"2026-06-05T06:46:15.197025399Z","DeletedAt":null,"paper_id":3084839,"paper_url":"https://arxiv.org/abs/2606.05687","paper_title":"Accelerating and Scaling MPC-Guided Reinforcement Learning for Humanoid Locomotion and Manipulation","repo_url":"https://github.com/junhengl/mpc-rl","is_official":false,"mentioned_in_paper":false,"mentioned_in_github":true,"github_stars":0}]}