{"ID":6138907,"CreatedAt":"2026-07-09T01:07:32.349475501Z","UpdatedAt":"2026-07-10T23:04:43.947021291Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2607.06856","arxiv_id":"2607.06856","title":"Gen4U: Unifying Video Generation and Understanding via Diffusion","abstract":"Prior work suggests that diffusion representations capture low-level geometry but struggle with high-level semantics. We demonstrate that state-of-the-art video diffusion models overcome this limitation. By systematically probing their intermediate activations using recent mutual-kNN alignment metrics, we reveal a highly structured latent space where visual representations evolve across both network depth and noise levels. We show that while moderate noise levels yield linearly separable global semantics, fine-grained details persist at lower noise levels but become spatially scattered, requiring attention mechanisms to decode. Building on these insights, we introduce Gen4U (Generation for Understanding), a framework that repurposes these generative representations with a single forward pass. Our experiments establish that frozen, large-scale video diffusion models function as highly competitive video encoders across a wide spectrum of tasks, spanning semantic and non-semantic objectives (video classification, depth estimation, camera pose estimation, image and video captioning). Bypassing fine-tuning, Gen4U unifies the generation and understanding paradigms, achieving strong perception performance while fully preserving the model's ability to generate high-quality video.","short_abstract":"Prior work suggests that diffusion representations capture low-level geometry but struggle with high-level semantics. We demonstrate that state-of-the-art video diffusion models overcome this limitation. By systematically probing their intermediate activations using recent mutual-kNN alignment metrics, we reveal a high...","url_abs":"https://arxiv.org/abs/2607.06856","url_pdf":"https://arxiv.org/pdf/2607.06856v1","authors":"[\"Michael King\",\"Aravindh Mahendran\",\"Matthew Koichi Grimes\",\"Fedor Kitashov\",\"Adham Elarabawy\",\"Pedro Velez\",\"Maks Ovsjanikov\",\"Viorica Pătrăucean\"]","published":"2026-07-07T23:17:26Z","proceeding":"cs.CV","tasks":"[\"cs.CV\",\"cs.LG\"]","methods":"[\"Diffusion Model\"]","has_code":false}