{"ID":5554324,"CreatedAt":"2026-07-02T02:11:27.934456424Z","UpdatedAt":"2026-07-04T16:50:11.910852832Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2607.01166","arxiv_id":"2607.01166","title":"Structured 4D Latent Predictive Model for Robot Planning","abstract":"Video predictive models are emerging as a powerful paradigm in robotics, offering a promising path toward task generalization, long-horizon planning, and flexible decision-making. However, prevailing approaches often operate on 2D video sequences, inherently lacking the 3D geometric understanding necessary for precise spatial reasoning and physical consistency. We introduce a Structured 4D Latent Predictive Model, which predicts the evolution of a scene's 3D structure in a structured latent space conditioned on observations and textual instructions. Our representation encodes the scene holistically and can be decoded into diverse 3D formats, enabling a more complete and 3D consistent scene understanding. This structured 4D latent predictive model serves as a planner, generating future scenes that are translated into executable actions by a goal-conditioned inverse dynamics module. Experiments demonstrate that our model generates futures with strong visual quality, substantially better 3D consistency and multi-view coherence compared to state-of-the-art video-based planners. Consequently, our full planning pipeline achieves superior performance on complex manipulation tasks, exhibits robust generalization to novel visual conditions, and proves effective on real-world robotic platforms. Our website is available at https://structured-4d-model.github.io/.","short_abstract":"Video predictive models are emerging as a powerful paradigm in robotics, offering a promising path toward task generalization, long-horizon planning, and flexible decision-making. However, prevailing approaches often operate on 2D video sequences, inherently lacking the 3D geometric understanding necessary for precise...","url_abs":"https://arxiv.org/abs/2607.01166","url_pdf":"https://arxiv.org/pdf/2607.01166v1","authors":"[\"Zhiyi Li\",\"Peilin Wu\",\"Xiaoshen Han\",\"Ruojin Cai\",\"Yilun Du\"]","published":"2026-07-01T16:52:49Z","proceeding":"cs.RO","tasks":"[\"cs.RO\",\"cs.CV\"]","methods":"[]","has_code":false}