{"ID":2842156,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2511.10087","arxiv_id":"2511.10087","title":"Opinion: Towards Unified Expressive Policy Optimization for Robust Robot Learning","abstract":"Offline-to-online reinforcement learning (O2O-RL) has emerged as a promising paradigm for safe and efficient robotic policy deployment but suffers from two fundamental challenges: limited coverage of multimodal behaviors and distributional shifts during online adaptation. We propose UEPO, a unified generative framework inspired by large language model pretraining and fine-tuning strategies. Our contributions are threefold: (1) a multi-seed dynamics-aware diffusion policy that efficiently captures diverse modalities without training multiple models; (2) a dynamic divergence regularization mechanism that enforces physically meaningful policy diversity; and (3) a diffusion-based data augmentation module that enhances dynamics model generalization. On the D4RL benchmark, UEPO achieves +5.9\\% absolute improvement over Uni-O4 on locomotion tasks and +12.4\\% on dexterous manipulation, demonstrating strong generalization and scalability.","short_abstract":"Offline-to-online reinforcement learning (O2O-RL) has emerged as a promising paradigm for safe and efficient robotic policy deployment but suffers from two fundamental challenges: limited coverage of multimodal behaviors and distributional shifts during online adaptation. We propose UEPO, a unified generative framework...","url_abs":"https://arxiv.org/abs/2511.10087","url_pdf":"https://arxiv.org/pdf/2511.10087v1","authors":"[\"Haidong Huang\",\"Haiyue Zhu. Jiayu Song\",\"Xixin Zhao\",\"Yaohua Zhou\",\"Jiayi Zhang\",\"Yuze Zhai\",\"Xiaocong Li\"]","published":"2025-11-13T08:42:20Z","proceeding":"cs.RO","tasks":"[\"cs.RO\",\"cs.AI\",\"cs.LG\"]","methods":"[\"Reinforcement Learning\",\"Diffusion Model\",\"Language Model\"]","has_code":false}