{"ID":6024181,"CreatedAt":"2026-07-08T01:00:23.257252134Z","UpdatedAt":"2026-07-10T00:23:02.025452025Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2607.05711","arxiv_id":"2607.05711","title":"FourTune: Towards Fully 4-Bit Efficient Post-Training for Diffusion Models","abstract":"Diffusion models have become a dominant paradigm for high-quality generative modeling, while post-training is essential for adapting them to diverse downstream applications. However, post-training of large diffusion models is still challenging due to the prohibitive memory footprints and slow training speed, which existing parameter-efficient fine-tuning methods only partially address. To overcome these limitations, we propose FourTune, an efficient post-training framework for diffusion models based on an end-to-end W4A4G4 paradigm. FourTune introduces a triple-branch hybrid pipeline that augments the standard LoRA architecture with a frozen numerical stabilizer to isolate quantization-sensitive outliers, enabling stable training under native 4-bit computation. In addition, FourTune employs hardware-efficient block-wise quantization and customized fused kernels to support efficient quantized backpropagation and reduce memory bandwidth overhead. Across customization, reinforcement learning, and distillation tasks, FourTune matches the quality of full-precision fine-tuning. On FLUX.1-dev (12B), FourTune reduces memory overhead by 2.25$\\times$ and increases end-to-end training throughput by 2.27$\\times$ compared to BF16 LoRA.","short_abstract":"Diffusion models have become a dominant paradigm for high-quality generative modeling, while post-training is essential for adapting them to diverse downstream applications. However, post-training of large diffusion models is still challenging due to the prohibitive memory footprints and slow training speed, which exis...","url_abs":"https://arxiv.org/abs/2607.05711","url_pdf":"https://arxiv.org/pdf/2607.05711v1","authors":"[\"Bowen Xue\",\"Zihan Min\",\"Xingyang Li\",\"Zhekai Zhang\",\"Haocheng Xi\",\"Lvmin Zhang\",\"Maneesh Agrawala\",\"Jun-Yan Zhu\",\"Song Han\",\"Yujun Lin\",\"Muyang Li\"]","published":"2026-07-07T00:34:30Z","proceeding":"cs.LG","tasks":"[\"cs.LG\",\"cs.CV\"]","methods":"[\"Reinforcement Learning\",\"Diffusion Model\",\"LoRA\"]","has_code":false}