{"ID":2864593,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2510.03270","arxiv_id":"2510.03270","title":"CoDA: Coding LM via Diffusion Adaptation","abstract":"Diffusion language models promise bidirectional context and infilling capabilities that autoregressive coders lack, yet practical systems remain heavyweight. We introduce CoDA, a 1.7B-parameter diffusion coder trained on TPU with a fully open-source training pipeline. CoDA pairs large-scale diffusion pre-training with code-centric mid-training and instruction tuning, enabling confidence-guided sampling that keeps inference latency competitive. On Humaneval, MBPP, and EvalPlus, CoDA-1.7B-Instruct matches or surpasses diffusion models up to 7B parameters. Our release includes model checkpoints, evaluation harnesses, and TPU training pipelines to accelerate research on lightweight diffusion-based coding assistants.","short_abstract":"Diffusion language models promise bidirectional context and infilling capabilities that autoregressive coders lack, yet practical systems remain heavyweight. We introduce CoDA, a 1.7B-parameter diffusion coder trained on TPU with a fully open-source training pipeline. CoDA pairs large-scale diffusion pre-training with...","url_abs":"https://arxiv.org/abs/2510.03270","url_pdf":"https://arxiv.org/pdf/2510.03270v1","authors":"[\"Haolin Chen\",\"Shiyu Wang\",\"Can Qin\",\"Bo Pang\",\"Zuxin Liu\",\"Jielin Qiu\",\"Jianguo Zhang\",\"Yingbo Zhou\",\"Zeyuan Chen\",\"Ran Xu\",\"Shelby Heinecke\",\"Silvio Savarese\",\"Caiming Xiong\",\"Huan Wang\",\"Weiran Yao\"]","published":"2025-09-27T05:41:55Z","proceeding":"cs.LG","tasks":"[\"cs.LG\",\"cs.AI\"]","methods":"[\"Diffusion Model\",\"Language Model\"]","has_code":false}