{"ID":5443902,"CreatedAt":"2026-07-01T02:07:11.383974684Z","UpdatedAt":"2026-07-14T23:57:39.674630735Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2606.32020","arxiv_id":"2606.32020","title":"Cross-Space Distillation: Teaching One-Step Students with Modern Diffusion Teachers","abstract":"Modern one-step diffusion models achieve impressive quality through distribution-based timestep distillation. Yet, they rely on a critical assumption: Teacher and Student must inhabit the same latent space. This Shared-Space constraint prevents knowledge transfer from modern high-capacity Teachers (e.g., SD 3.5 and Flux) into compact, deployment-friendly Students such as SD 1.5, whose latent resolution and VAE parameterization differ from the Teacher. We formalize this overlooked regime as Cross-Space Distillation, where Teacher and Student differ in both latent resolution and VAE space. To enable distillation under this mismatch, we introduce the Bridge, a lightweight latent interface that maps Student latents into the Teacher space without modifying the Student backbone. Bridge combines a frozen Student VAE decoder as a spatial prior with a compact learnable projector, and is trained with latent reconstruction and attention fidelity objectives for stable Teacher-space alignment. Across diverse modern Teachers, Bridge enables substantial gains for compact one-step Students; for example, it improves SD 1.5 from 5.4 to 9.4 HPSv3 while preserving one-step inference, low latency, and broad ecosystem compatibility. These results show that heterogeneous large Teachers can be distilled into efficient, deployable backbones through a lightweight latent-space interface.","short_abstract":"Modern one-step diffusion models achieve impressive quality through distribution-based timestep distillation. Yet, they rely on a critical assumption: Teacher and Student must inhabit the same latent space. This Shared-Space constraint prevents knowledge transfer from modern high-capacity Teachers (e.g., SD 3.5 and Flu...","url_abs":"https://arxiv.org/abs/2606.32020","url_pdf":"https://arxiv.org/pdf/2606.32020v1","authors":"[\"Anh Nguyen\",\"Ngan Nguyen\",\"Duc Vu\",\"Trung Dao\",\"Viet Nguyen\",\"Quan Dao\",\"Kien Nguyen\",\"Chi Tran\",\"Phong Nguyen\",\"Khoi Nguyen\",\"Cuong Pham\",\"Dimitris Metaxas\",\"Vishal M. Patel\",\"Anh Tran\"]","published":"2026-06-30T17:51:26Z","proceeding":"cs.CV","tasks":"[\"cs.CV\"]","methods":"[\"Diffusion Model\",\"Variational Autoencoder\"]","has_code":false}