{"ID":2868399,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2509.16622","arxiv_id":"2509.16622","title":"Audio-Conditioned Diffusion LLMs for ASR and Deliberation Processing","abstract":"Diffusion-based large language models (DLLMs) have recently attracted growing interest as an alternative to autoregressive decoders. In this work, we present an empirical study on using the diffusion-based large language model LLaDA for automatic speech recognition (ASR). We first investigate its use as an external deliberation-based processing module for Whisper-LLaMA transcripts. By leveraging the bidirectional attention and denoising capabilities of LLaDA, we explore random masking, low-confidence masking, and semi-autoregressive strategies, showing that Whisper-LLaDA substantially reduces WER compared with the baseline. On LibriSpeech, the best cascade system achieves 2.25%/4.94% WER on test-clean/test-other, representing a 12.3% relative improvement over the Whisper-LLaMA baseline on the test-other split. In contrast, a plain-text LLaDA without acoustic features fails to improve accuracy, highlighting the importance of audio-conditioned embeddings. We further evaluate Whisper-LLaDA as a standalone decoder for ASR with diffusion-based and semi-autoregressive decoding. Most experimental configurations achieve faster inference than the Whisper-LLaMA baseline, although recognition accuracy is slightly lower. These findings offer an empirical view of diffusion-based LLMs for ASR and point to promising directions for improvements. Code and model are open-sourced at https://github.com/liuzhan22/Diffusion-ASR.","short_abstract":"Diffusion-based large language models (DLLMs) have recently attracted growing interest as an alternative to autoregressive decoders. In this work, we present an empirical study on using the diffusion-based large language model LLaDA for automatic speech recognition (ASR). We first investigate its use as an external del...","url_abs":"https://arxiv.org/abs/2509.16622","url_pdf":"https://arxiv.org/pdf/2509.16622v3","authors":"[\"Mengqi Wang\",\"Zhan Liu\",\"Zengrui Jin\",\"Guangzhi Sun\",\"Chao Zhang\",\"Philip C. Woodland\"]","published":"2025-09-20T10:48:06Z","proceeding":"eess.AS","tasks":"[\"eess.AS\",\"cs.AI\",\"cs.SD\"]","methods":"[\"Diffusion Model\",\"Large Language Model\",\"Language Model\"]","has_code":false,"code_links":[{"ID":609581,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_id":2868399,"paper_url":"https://arxiv.org/abs/2509.16622","paper_title":"Audio-Conditioned Diffusion LLMs for ASR and Deliberation Processing","repo_url":"https://github.com/liuzhan22/Diffusion-ASR","is_official":false,"mentioned_in_paper":false,"mentioned_in_github":true,"github_stars":0}]}