{"ID":6620649,"CreatedAt":"2026-07-15T01:01:48.440468303Z","UpdatedAt":"2026-07-17T07:22:21.663090451Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2607.12706","arxiv_id":"2607.12706","title":"AutoSIFT: Automatic Style Sifting for Controllable Speech Generation with Arbitrary Style Infilling","abstract":"State-of-the-art text-to-speech (TTS) models achieve impressive naturalness and expressiveness, yet fine-grained, disentangled control over speaking styles remains challenging. In professional scenarios such as film dubbing, game voice acting, and video content generation, users often need to modify a specific style category, such as emotion, age, or gender, while preserving all others. Existing style-controllable TTS methods typically rely on either text-described styles or speech-reference style transfer, making it difficult to jointly control explicit semantic attributes and preserve subtle, text-undescribed prosodic details. We propose AutoSIFT, a controllable speech generation framework for category-level style editing. AutoSIFT decomposes speaking style into known text-describable categories and unknown residual styles that capture non-verbal prosody and speaker-specific nuances. It consists of a generalized Style Disentangler, which extracts category-aware style prototypes from reference speech, and an Arbitrary Style Infiller, which selectively infills unspecified style categories from the reference. By replacing only text-specified style categories while preserving residual speech-derived styles, AutoSIFT enables natural, expressive, and highly customizable speech generation.","short_abstract":"State-of-the-art text-to-speech (TTS) models achieve impressive naturalness and expressiveness, yet fine-grained, disentangled control over speaking styles remains challenging. In professional scenarios such as film dubbing, game voice acting, and video content generation, users often need to modify a specific style ca...","url_abs":"https://arxiv.org/abs/2607.12706","url_pdf":"https://arxiv.org/pdf/2607.12706v1","authors":"[\"Haowei Lou\",\"Junda Wu\",\"Chengkai Huang\",\"Tong Yu\",\"Hye-young Paik\",\"Wen Hu\",\"Lina Yao\"]","published":"2026-07-14T12:28:43Z","proceeding":"cs.SD","tasks":"[\"cs.SD\"]","methods":"[]","has_code":false}