{"ID":5937681,"CreatedAt":"2026-07-07T03:14:33.014478982Z","UpdatedAt":"2026-07-08T12:38:41.542637154Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2607.04276","arxiv_id":"2607.04276","title":"EMPURPLE: A Free Lunch for Diffusion Distillation based on the Information Bottleneck","abstract":"Diffusion models achieve impressive image-generation quality but remain expensive at inference time. Diffusion distillation reduces sampling steps, yet many distilled models, including SDXL-Lightning and distribution matching distillation methods, suffer from degraded Fréchet Inception Distance (FID). We analyze this phenomenon through a PAC-style generalization bound. Our analysis suggests that aggressive early-step redirection of the velocity field makes the distillation target harder to learn, enlarging the train-test gap. As a result, early-step output distributions differ between training and inference, causing distribution mismatch in the intermediate noisy latent used as next-step inputs. We empirically validate this mechanism by showing reduced diversity in both intermediate features and final outputs. To address this issue, we propose EMPURPLE, a simple training-free method that recycles intermediate latents sampled from the original model. EMPURPLE is model-agnostic and improves FID by 7\\% to 20\\% across DMD2, Hyper-SD, FlashSD, and SDXL-Lightning. The repo is: https://github.com/TheLovesOfLadyPurple/Empurple-Training-Free-Algorithm-To-enhance-Diversity-of-The-Diffusion-Distillation-Model","short_abstract":"Diffusion models achieve impressive image-generation quality but remain expensive at inference time. Diffusion distillation reduces sampling steps, yet many distilled models, including SDXL-Lightning and distribution matching distillation methods, suffer from degraded Fréchet Inception Distance (FID). We analyze this p...","url_abs":"https://arxiv.org/abs/2607.04276","url_pdf":"https://arxiv.org/pdf/2607.04276v1","authors":"[\"Zilai Li\",\"Lujia Bai\"]","published":"2026-07-05T12:40:51Z","proceeding":"cs.CV","tasks":"[\"cs.CV\"]","methods":"[\"Diffusion Model\"]","has_code":false,"code_links":[{"ID":613980,"CreatedAt":"2026-07-07T03:14:33.014478982Z","UpdatedAt":"2026-07-07T03:14:33.014478982Z","DeletedAt":null,"paper_id":5937681,"paper_url":"https://arxiv.org/abs/2607.04276","paper_title":"EMPURPLE: A Free Lunch for Diffusion Distillation based on the Information Bottleneck","repo_url":"https://github.com/TheLovesOfLadyPurple/Empurple-Training-Free-Algorithm-To-enhance-Diversity-of-The-Diffusion-Distillation-Model","is_official":false,"mentioned_in_paper":false,"mentioned_in_github":true,"github_stars":0}]}