{"ID":2837658,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2511.19269","arxiv_id":"2511.19269","title":"CDLM: Consistency Diffusion Language Models For Faster Sampling","abstract":"Diffusion Language Models (DLMs) offer a promising parallel generation paradigm but suffer from slow inference due to numerous refinement steps and the inability to use standard KV caching. We introduce CDLM (Consistency Diffusion Language Models), a training-based acceleration method that simultaneously tackles both bottlenecks. CDLM integrates consistency modeling to drastically reduce the number of required sampling steps by enabling multi-token finalization. Furthermore, we enforce a block-wise causal attention mask during fine-tuning, making the model fully compatible with KV caching. Experiments show CDLM achieves 3.6x-14.5x lower latency while maintaining competitive accuracy on math and coding tasks. The full training and evaluation code is available at https://github.com/SqueezeAILab/CDLM.","short_abstract":"Diffusion Language Models (DLMs) offer a promising parallel generation paradigm but suffer from slow inference due to numerous refinement steps and the inability to use standard KV caching. We introduce CDLM (Consistency Diffusion Language Models), a training-based acceleration method that simultaneously tackles both b...","url_abs":"https://arxiv.org/abs/2511.19269","url_pdf":"https://arxiv.org/pdf/2511.19269v2","authors":"[\"Minseo Kim\",\"Chenfeng Xu\",\"Coleman Hooper\",\"Harman Singh\",\"Ben Athiwaratkun\",\"Ce Zhang\",\"Kurt Keutzer\",\"Amir Gholami\"]","published":"2025-11-24T16:21:25Z","proceeding":"cs.LG","tasks":"[\"cs.LG\",\"cs.CL\"]","methods":"[\"Diffusion Model\",\"Language Model\"]","has_code":false,"code_links":[{"ID":606706,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_id":2837658,"paper_url":"https://arxiv.org/abs/2511.19269","paper_title":"CDLM: Consistency Diffusion Language Models For Faster Sampling","repo_url":"https://github.com/SqueezeAILab/CDLM","is_official":false,"mentioned_in_paper":false,"mentioned_in_github":true,"github_stars":0}]}