{"ID":2879091,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2508.16939","arxiv_id":"2508.16939","title":"Sig-DEG for Distillation: Making Diffusion Models Faster and Lighter","abstract":"Diffusion models have achieved state-of-the-art results in generative modelling but remain computationally intensive at inference time, often requiring thousands of discretization steps. To this end, we propose Sig-DEG (Signature-based Differential Equation Generator), a novel generator for distilling pre-trained diffusion models, which can universally approximate the backward diffusion process at a coarse temporal resolution. Inspired by high-order approximations of stochastic differential equations (SDEs), Sig-DEG leverages partial signatures to efficiently summarize Brownian motion over sub-intervals and adopts a recurrent structure to enable accurate global approximation of the SDE solution. Distillation is formulated as a supervised learning task, where Sig-DEG is trained to match the outputs of a fine-resolution diffusion model on a coarse time grid. During inference, Sig-DEG enables fast generation, as the partial signature terms can be simulated exactly without requiring fine-grained Brownian paths. Experiments demonstrate that Sig-DEG achieves competitive generation quality while reducing the number of inference steps by an order of magnitude. Our results highlight the effectiveness of signature-based approximations for efficient generative modeling.","short_abstract":"Diffusion models have achieved state-of-the-art results in generative modelling but remain computationally intensive at inference time, often requiring thousands of discretization steps. To this end, we propose Sig-DEG (Signature-based Differential Equation Generator), a novel generator for distilling pre-trained diffu...","url_abs":"https://arxiv.org/abs/2508.16939","url_pdf":"https://arxiv.org/pdf/2508.16939v1","authors":"[\"Lei Jiang\",\"Wen Ge\",\"Niels Cariou-Kotlarek\",\"Mingxuan Yi\",\"Po-Yu Chen\",\"Lingyi Yang\",\"Francois Buet-Golfouse\",\"Gaurav Mittal\",\"Hao Ni\"]","published":"2025-08-23T08:16:14Z","proceeding":"cs.LG","tasks":"[\"cs.LG\",\"math.PR\",\"stat.ML\"]","methods":"[\"Diffusion Model\"]","has_code":false}