{"ID":5938103,"CreatedAt":"2026-07-07T03:14:33.014478982Z","UpdatedAt":"2026-07-07T04:02:55.758908856Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2607.02642","arxiv_id":"2607.02642","title":"GigaWorld-1: A Roadmap to Build World Models for Robot Policy Evaluation","abstract":"Evaluating embodied robot foundation models remains a critical bottleneck; unlike large language models efficiently assessed via digital benchmarks, robotic policies require slow, costly real-world rollouts limited by hardware and human supervision, which has driven interest in world models as surrogate policy evaluators, yet the key properties that make a world model reliable for policy assessment remain poorly understood. This work presents a systematic study of world models for robotic policy evaluation and introduces WMBench, a benchmark constructed from real-robot teleoperation data and matched policy rollouts covering diverse manipulation tasks to enable controlled comparisons across model families, action encodings, rollout horizons, and evaluation metrics. Using WMBench, we analyze 7 video world models, 4 action representation schemes, and over 324,000 simulated policy rollouts paired with real robot executions, further enriching our analysis with large-scale community submissions from the CVPR 2026 GigaBrain Challenge, curated synthetic trajectories, and a training videos spanning more than 12,000 hours. Our experiments deliver three core insights: evaluator quality is dominated by long-horizon, action-faithful rollout consistency rather than short-term visual realism; pretraining gains stem not only from data scale but from balancing general world knowledge with robot-specific controllability; and architectural choices including action encoding, memory design, and evaluator-focused post-training strongly determine alignment with real-world robot behavior. Drawing on these results, we derive a practical design roadmap and realize it in \\textit{GigaWorld-1}, a world model specially optimized for policy evaluation, and we fully release our code, models, datasets, and toolkits to advance scalable evaluation research for embodied foundation models.","short_abstract":"Evaluating embodied robot foundation models remains a critical bottleneck; unlike large language models efficiently assessed via digital benchmarks, robotic policies require slow, costly real-world rollouts limited by hardware and human supervision, which has driven interest in world models as surrogate policy evaluato...","url_abs":"https://arxiv.org/abs/2607.02642","url_pdf":"https://arxiv.org/pdf/2607.02642v1","authors":"[\"GigaWorld Team\",\"Angyuan Ma\",\"Boyuan Wang\",\"Bohan Li\",\"Chaojun Ni\",\"Guo Li\",\"Guan Huang\",\"Guosheng Zhao\",\"Hao Li\",\"Hengtao Li\",\"Jingyu Liu\",\"Jiwen Lu\",\"Qiuping Deng\",\"Tingdong Yu\",\"Xuancheng Xu\",\"Xinyu Zhou\",\"Xiuwei Xu\",\"Xinze Chen\",\"Xiaofeng Wang\",\"Xiaoyu Tian\",\"Yang Wang\",\"Yifan Chang\",\"Yukun Zhou\",\"Yun Ye\",\"Zhenyu Wu\",\"Zhanqian Wu\",\"Zheng Zhu\"]","published":"2026-07-02T17:02:43Z","proceeding":"cs.RO","tasks":"[\"cs.RO\"]","methods":"[\"Language Model\"]","has_code":false}