{"ID":2884803,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2508.06391","arxiv_id":"2508.06391","title":"Improved Dysarthric Speech to Text Conversion via TTS Personalization","abstract":"We present a case study on developing a customized speech-to-text system for a Hungarian speaker with severe dysarthria. State-of-the-art automatic speech recognition (ASR) models struggle with zero-shot transcription of dysarthric speech, yielding high error rates. To improve performance with limited real dysarthric data, we fine-tune an ASR model using synthetic speech generated via a personalized text-to-speech (TTS) system. We introduce a method for generating synthetic dysarthric speech with controlled severity by leveraging premorbidity recordings of the given speaker and speaker embedding interpolation, enabling ASR fine-tuning on a continuum of impairments. Fine-tuning on both real and synthetic dysarthric speech reduces the character error rate (CER) from 36-51% (zero-shot) to 7.3%. Our monolingual FastConformer_Hu ASR model significantly outperforms Whisper-turbo when fine-tuned on the same data, and the inclusion of synthetic speech contributes to an 18% relative CER reduction. These results highlight the potential of personalized ASR systems for improving accessibility for individuals with severe speech impairments.","short_abstract":"We present a case study on developing a customized speech-to-text system for a Hungarian speaker with severe dysarthria. State-of-the-art automatic speech recognition (ASR) models struggle with zero-shot transcription of dysarthric speech, yielding high error rates. To improve performance with limited real dysarthric d...","url_abs":"https://arxiv.org/abs/2508.06391","url_pdf":"https://arxiv.org/pdf/2508.06391v1","authors":"[\"Péter Mihajlik\",\"Éva Székely\",\"Piroska Barta\",\"Máté Soma Kádár\",\"Gergely Dobsinszki\",\"László Tóth\"]","published":"2025-08-08T15:21:29Z","proceeding":"cs.SD","tasks":"[\"cs.SD\",\"cs.HC\"]","methods":"[]","has_code":false}