{"ID":2921192,"CreatedAt":"2026-06-02T02:42:49.606572591Z","UpdatedAt":"2026-06-04T00:54:56.190393508Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2606.01677","arxiv_id":"2606.01677","title":"UniVocal: Unified Speech-Singing Code-Switching Synthesis","abstract":"We propose UniVocal, a unified framework that implicitly infers vocal modes from text context to pioneer Speech-Singing Code-Switching (SCS) Synthesis - a task where transitions are autonomously driven by textual semantics, akin to seamless human language blending. Unlike single-mode generation or systems relying on switching-control tags, our proposed UniVocal implicitly infers vocal modes solely from text context. To achieve this, we employ a data-efficient two-stage curriculum learning strategy that progressively trains a competitive TTS system to acquire the desired SCS capability. Addressing data scarcity, we introduce a scalable pipeline to synthesize diverse code-switching data that is both semantically and acoustically natural, alongside a new multi-scenario benchmark, SCSBench. To address limitations of semantic tokenizers in capturing acoustic details, we also introduce refined cent token and Chain-of-Thought (CoT) generation for planning prosody before content generation, effectively enhancing empathetic speech generation and singing melody. Experimental results demonstrate that UniVocal achieves state-of-the-art performance on SCSBench while maintaining competitive performance on regular speech and singing tasks. Audio samples are available at https://project-univocal-demo.github.io/demo/. The code and dataset are released at https://github.com/FunAudioLLM/FunResearch/tree/main/UniVocal.","short_abstract":"We propose UniVocal, a unified framework that implicitly infers vocal modes from text context to pioneer Speech-Singing Code-Switching (SCS) Synthesis - a task where transitions are autonomously driven by textual semantics, akin to seamless human language blending. Unlike single-mode generation or systems relying on sw...","url_abs":"https://arxiv.org/abs/2606.01677","url_pdf":"https://arxiv.org/pdf/2606.01677v1","authors":"[\"Yufei Shi\",\"Qian Chen\",\"Wen Wang\",\"Xiangang Li\",\"Zhen-Hua Ling\",\"Yang Ai\"]","published":"2026-06-01T04:35:28Z","proceeding":"cs.SD","tasks":"[\"cs.SD\"]","methods":"[\"Large Language Model\"]","has_code":false,"code_links":[{"ID":612572,"CreatedAt":"2026-06-02T02:42:49.606572591Z","UpdatedAt":"2026-06-02T02:42:49.606572591Z","DeletedAt":null,"paper_id":2921192,"paper_url":"https://arxiv.org/abs/2606.01677","paper_title":"UniVocal: Unified Speech-Singing Code-Switching Synthesis","repo_url":"https://github.com/FunAudioLLM/FunResearch","is_official":false,"mentioned_in_paper":false,"mentioned_in_github":true,"github_stars":0}]}