{"ID":2831823,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2512.07831","arxiv_id":"2512.07831","title":"UnityVideo: Unified Multi-Modal Multi-Task Learning for Enhancing World-Aware Video Generation","abstract":"Recent video generation models demonstrate impressive synthesis capabilities but remain limited by single-modality conditioning, constraining their holistic world understanding. This stems from insufficient cross-modal interaction and limited modal diversity for comprehensive world knowledge representation. To address these limitations, we introduce UnityVideo, a unified framework for world-aware video generation that jointly learns across multiple modalities (segmentation masks, human skeletons, DensePose, optical flow, and depth maps) and training paradigms. Our approach features two core components: (1) dynamic noising to unify heterogeneous training paradigms, and (2) a modality switcher with an in-context learner that enables unified processing via modular parameters and contextual learning. We contribute a large-scale unified dataset with 1.3M samples. Through joint optimization, UnityVideo accelerates convergence and significantly enhances zero-shot generalization to unseen data. We demonstrate that UnityVideo achieves superior video quality, consistency, and improved alignment with physical world constraints. Code and data can be found at: https://github.com/dvlab-research/UnityVideo","short_abstract":"Recent video generation models demonstrate impressive synthesis capabilities but remain limited by single-modality conditioning, constraining their holistic world understanding. This stems from insufficient cross-modal interaction and limited modal diversity for comprehensive world knowledge representation. To address...","url_abs":"https://arxiv.org/abs/2512.07831","url_pdf":"https://arxiv.org/pdf/2512.07831v1","authors":"[\"Jiehui Huang\",\"Yuechen Zhang\",\"Xu He\",\"Yuan Gao\",\"Zhi Cen\",\"Bin Xia\",\"Yan Zhou\",\"Xin Tao\",\"Pengfei Wan\",\"Jiaya Jia\"]","published":"2025-12-08T18:59:01Z","proceeding":"cs.CV","tasks":"[\"cs.CV\"]","methods":"[]","has_code":false,"code_links":[{"ID":606171,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_id":2831823,"paper_url":"https://arxiv.org/abs/2512.07831","paper_title":"UnityVideo: Unified Multi-Modal Multi-Task Learning for Enhancing World-Aware Video Generation","repo_url":"https://github.com/dvlab-research/UnityVideo","is_official":false,"mentioned_in_paper":false,"mentioned_in_github":true,"github_stars":0}]}