{"ID":2894631,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2507.11554","arxiv_id":"2507.11554","title":"Inversion-DPO: Precise and Efficient Post-Training for Diffusion Models","abstract":"Recent advancements in diffusion models (DMs) have been propelled by alignment methods that post-train models to better conform to human preferences. However, these approaches typically require computation-intensive training of a base model and a reward model, which not only incurs substantial computational overhead but may also compromise model accuracy and training efficiency. To address these limitations, we propose Inversion-DPO, a novel alignment framework that circumvents reward modeling by reformulating Direct Preference Optimization (DPO) with DDIM inversion for DMs. Our method conducts intractable posterior sampling in Diffusion-DPO with the deterministic inversion from winning and losing samples to noise and thus derive a new post-training paradigm. This paradigm eliminates the need for auxiliary reward models or inaccurate appromixation, significantly enhancing both precision and efficiency of training. We apply Inversion-DPO to a basic task of text-to-image generation and a challenging task of compositional image generation. Extensive experiments show substantial performance improvements achieved by Inversion-DPO compared to existing post-training methods and highlight the ability of the trained generative models to generate high-fidelity compositionally coherent images. For the post-training of compostitional image geneation, we curate a paired dataset consisting of 11,140 images with complex structural annotations and comprehensive scores, designed to enhance the compositional capabilities of generative models. Inversion-DPO explores a new avenue for efficient, high-precision alignment in diffusion models, advancing their applicability to complex realistic generation tasks. Our code is available at https://github.com/MIGHTYEZ/Inversion-DPO","short_abstract":"Recent advancements in diffusion models (DMs) have been propelled by alignment methods that post-train models to better conform to human preferences. However, these approaches typically require computation-intensive training of a base model and a reward model, which not only incurs substantial computational overhead bu...","url_abs":"https://arxiv.org/abs/2507.11554","url_pdf":"https://arxiv.org/pdf/2507.11554v4","authors":"[\"Zejian Li\",\"Yize Li\",\"Chenye Meng\",\"Zhongni Liu\",\"Yang Ling\",\"Shengyuan Zhang\",\"Guang Yang\",\"Changyuan Yang\",\"Zhiyuan Yang\",\"Lingyun Sun\"]","published":"2025-07-14T02:59:28Z","proceeding":"cs.CV","tasks":"[\"cs.CV\",\"cs.AI\"]","methods":"[\"Diffusion Model\"]","has_code":false,"code_links":[{"ID":612117,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_id":2894631,"paper_url":"https://arxiv.org/abs/2507.11554","paper_title":"Inversion-DPO: Precise and Efficient Post-Training for Diffusion Models","repo_url":"https://github.com/MIGHTYEZ/Inversion-DPO","is_official":false,"mentioned_in_paper":false,"mentioned_in_github":true,"github_stars":0}]}