{"ID":5937763,"CreatedAt":"2026-07-07T03:14:33.014478982Z","UpdatedAt":"2026-07-10T01:27:54.063497417Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2607.04434","arxiv_id":"2607.04434","title":"RoboDojo: A Unified Sim-and-Real Benchmark for Comprehensive Evaluation of Generalist Robot Manipulation Policies","abstract":"Generalist robot manipulation policies have advanced rapidly, yet existing benchmarks remain limited in systematically evaluating their capabilities. Many rely on simple, short-horizon, or skill-narrow tasks with limited capability coverage, and are often conducted only in simulation or only in the real world. Simulation enables scalable feedback but misses physical deployment challenges, while real-world evaluation is costly, time-consuming, and difficult to reproduce. We introduce RoboDojo, a unified sim-and-real benchmark for comprehensive evaluation of generalist robot manipulation policies. RoboDojo includes 42 simulation tasks and 18 real-world tasks covering diverse and complementary manipulation capabilities. The simulation benchmark evaluates five dimensions: generalization, memory, precision, long-horizon execution, and open-vocabulary instruction following, while the real-world benchmark exposes policies to challenging physical-world deployment conditions. RoboDojo supports scalable evaluation through heterogeneous parallel simulation in Isaac Sim and provides RoboDojo-RealEval, a reproducible real-world evaluation system with remote cloud access, standardized hardware, scene reset, evaluation protocol, and deployment interface. Together with XPolicyLab, policies can be integrated once and evaluated across simulation and real-world settings with minimal adaptation. We integrate 30 policies into XPolicyLab and evaluate them on RoboDojo, establishing a public leaderboard and systematic analysis of current policy performance. The website is available at http://robodojo-benchmark.com/.","short_abstract":"Generalist robot manipulation policies have advanced rapidly, yet existing benchmarks remain limited in systematically evaluating their capabilities. Many rely on simple, short-horizon, or skill-narrow tasks with limited capability coverage, and are often conducted only in simulation or only in the real world. Simulati...","url_abs":"https://arxiv.org/abs/2607.04434","url_pdf":"https://arxiv.org/pdf/2607.04434v1","authors":"[\"Tianxing Chen\",\"Yue Chen\",\"Zixuan Li\",\"Junyuan Tang\",\"Kailun Su\",\"Weijie Wan\",\"Baijun Chen\",\"Haoran Lu\",\"Haowen Yan\",\"Honghao Su\",\"Zhiyang Dou\",\"Kaixuan Wang\",\"Dandan Zhang\",\"Yunze Liu\",\"Yan Qin\",\"Qiwei Liang\",\"Qiwei Wu\",\"Zijian Lin\",\"Wenwei Lin\",\"Yuran Wang\",\"Minghua He\",\"Tianshu Wu\",\"Ruihai Wu\",\"Jingquan Zhou\",\"Kai-Chong Lei\",\"Haibao Yu\",\"Yuanfeng Ji\",\"Weiyang Jin\",\"Guanyu Lin\",\"Xiaofan Li\",\"Qi Xiong\",\"Renjing Xu\",\"Zhongyu Li\",\"Wenhao Chai\",\"Enze Xie\",\"Ziwei Wang\",\"Yao Mu\",\"Hao Dong\",\"Wojciech Matusik\",\"Mingyu Ding\",\"Wenbo Ding\",\"Ping Luo\",\"Masayoshi Tomizuka\"]","published":"2026-07-05T17:58:02Z","proceeding":"cs.RO","tasks":"[\"cs.RO\",\"cs.AI\",\"cs.CV\",\"cs.GR\"]","methods":"[]","project_urls":"[\"http://robodojo-benchmark.com/\"]","has_code":false}