{"ID":2826357,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2512.19583","arxiv_id":"2512.19583","title":"Learning Generalizable Hand-Object Tracking from Synthetic Demonstrations","abstract":"We present a system for learning generalizable hand-object tracking controllers purely from synthetic data, without requiring any human demonstrations. Our approach makes two key contributions: (1) HOP, a Hand-Object Planner, which can synthesize diverse hand-object trajectories; and (2) HOT, a Hand-Object Tracker that bridges synthetic-to-physical transfer through reinforcement learning and interaction imitation learning, delivering a generalizable controller conditioned on target hand-object states. Our method extends to diverse object shapes and hand morphologies. Through extensive evaluations, we show that our approach enables dexterous hands to track challenging, long-horizon sequences including object re-arrangement and agile in-hand reorientation. These results represent a significant step toward scalable foundation controllers for manipulation that can learn entirely from synthetic data, breaking the data bottleneck that has long constrained progress in dexterous manipulation.","short_abstract":"We present a system for learning generalizable hand-object tracking controllers purely from synthetic data, without requiring any human demonstrations. Our approach makes two key contributions: (1) HOP, a Hand-Object Planner, which can synthesize diverse hand-object trajectories; and (2) HOT, a Hand-Object Tracker that...","url_abs":"https://arxiv.org/abs/2512.19583","url_pdf":"https://arxiv.org/pdf/2512.19583v1","authors":"[\"Yinhuai Wang\",\"Runyi Yu\",\"Hok Wai Tsui\",\"Xiaoyi Lin\",\"Hui Zhang\",\"Qihan Zhao\",\"Ke Fan\",\"Miao Li\",\"Jie Song\",\"Jingbo Wang\",\"Qifeng Chen\",\"Ping Tan\"]","published":"2025-12-22T17:08:54Z","proceeding":"cs.RO","tasks":"[\"cs.RO\",\"cs.GR\"]","methods":"[\"Reinforcement Learning\"]","has_code":false}