{"ID":2828932,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2512.13251","arxiv_id":"2512.13251","title":"DisCo-Speech: Controllable Zero-Shot Speech Generation with A Disentangled Speech Codec","abstract":"Codec-based language models (LMs) have revolutionized text-to-speech (TTS). However, standard codecs entangle timbre and prosody, which hinders independent control in continuation-based LMs. To tackle this challenge, we propose DisCo-Speech, a zero-shot controllable TTS framework featuring a disentangled speech codec (DisCodec) and an LM-based generator. The core component DisCodec employs a two-stage design: 1) tri-factor disentanglement to separate speech into content, prosody, and timbre subspaces via parallel encoders and hybrid losses; and 2) fusion and reconstruction that merges content and prosody into unified content-prosody tokens suitable for LM prediction, while jointly optimizing reconstruction to address the disentanglement-reconstruction trade-off. This allows the LM to perform prosodic continuation from a style prompt while the decoder injects target timbre, enabling flexible zero-shot control. Experiments demonstrate that DisCo-Speech achieves competitive voice cloning and superior zero-shot prosody control. By resolving the core entanglement at the codec level, DisCo-Speech provides a robust foundation for controllable speech synthesis.","short_abstract":"Codec-based language models (LMs) have revolutionized text-to-speech (TTS). However, standard codecs entangle timbre and prosody, which hinders independent control in continuation-based LMs. To tackle this challenge, we propose DisCo-Speech, a zero-shot controllable TTS framework featuring a disentangled speech codec (...","url_abs":"https://arxiv.org/abs/2512.13251","url_pdf":"https://arxiv.org/pdf/2512.13251v3","authors":"[\"Tao Li\",\"Wenshuo Ge\",\"Zhichao Wang\",\"Zihao Cui\",\"Yong Ma\",\"Yingying Gao\",\"Chao Deng\",\"Shilei Zhang\",\"Junlan Feng\"]","published":"2025-12-15T12:06:17Z","proceeding":"cs.SD","tasks":"[\"cs.SD\"]","methods":"[\"Language Model\"]","has_code":false}