{"ID":5438681,"CreatedAt":"2026-07-01T01:17:58.482524686Z","UpdatedAt":"2026-07-03T07:00:11.61005204Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2606.31232","arxiv_id":"2606.31232","title":"Delta-JEPA: Learning Action-Sensitive World Models via Latent Difference Decoding","abstract":"Learning visual world models for planning requires compact latent dynamics that remain sensitive to actions, yet reconstruction-free joint-embedding objectives can collapse to action-insensitive representations. We propose Delta-JEPA, an end-to-end reconstruction-free world model that augments latent forward prediction with a Latent Difference Action Decoder (LDAD). Unlike inverse decoders that infer actions from concatenated endpoint embeddings, LDAD reconstructs the executed action from the latent displacement between consecutive observations. This displacement-level supervision directly regularizes transition geometry: adjacent embeddings cannot collapse without losing action information, and different actions are encouraged to induce distinguishable latent changes for rollout-based planning. Delta-JEPA uses only latent prediction and action reconstruction, avoiding pixel reconstruction and distribution-matching regularizers. Across four visual continuous-control tasks, Delta-JEPA improves planning over JEPA-based and representation-learning world model baselines. Ablations show that displacement-based action decoding is consistently more effective than endpoint concatenation, and action-sensitivity analyses show clearer action-conditioned latent responses. These results indicate that supervising latent differences is a simple and effective mechanism for collapse-resistant and action-sensitive world model learning.","short_abstract":"Learning visual world models for planning requires compact latent dynamics that remain sensitive to actions, yet reconstruction-free joint-embedding objectives can collapse to action-insensitive representations. We propose Delta-JEPA, an end-to-end reconstruction-free world model that augments latent forward prediction...","url_abs":"https://arxiv.org/abs/2606.31232","url_pdf":"https://arxiv.org/pdf/2606.31232v1","authors":"[\"Zhenghao Zhang\",\"Yuanxiang Wang\",\"Zhenyu Guan\",\"Yujia Yang\",\"Bingkang Shi\",\"Tianyu Zong\",\"Hongzhu Yi\",\"Guoqing Chao\",\"Xingchen Chen\",\"Tiankun Yang\",\"Chenxi Bao\",\"Tao Yu\",\"Jingjing Zhou\",\"Jungang Xu\"]","published":"2026-06-30T07:08:24Z","proceeding":"cs.AI","tasks":"[\"cs.AI\"]","methods":"[]","has_code":false}