{"ID":2866596,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2509.20128","arxiv_id":"2509.20128","title":"KSDiff: Keyframe-Augmented Speech-Aware Dual-Path Diffusion for Facial Animation","abstract":"Audio-driven facial animation has made significant progress in multimedia applications, with diffusion models showing strong potential for talking-face synthesis. However, most existing works treat speech features as a monolithic representation and fail to capture their fine-grained roles in driving different facial motions, while also overlooking the importance of modeling keyframes with intense dynamics. To address these limitations, we propose KSDiff, a Keyframe-Augmented Speech-Aware Dual-Path Diffusion framework. Specifically, the raw audio and transcript are processed by a Dual-Path Speech Encoder (DPSE) to disentangle expression-related and head-pose-related features, while an autoregressive Keyframe Establishment Learning (KEL) module predicts the most salient motion frames. These components are integrated into a Dual-path Motion generator to synthesize coherent and realistic facial motions. Extensive experiments on HDTF and VoxCeleb demonstrate that KSDiff achieves state-of-the-art performance, with improvements in both lip synchronization accuracy and head-pose naturalness. Our results highlight the effectiveness of combining speech disentanglement with keyframe-aware diffusion for talking-head generation. The demo page is available at: https://kincin.github.io/KSDiff/.","short_abstract":"Audio-driven facial animation has made significant progress in multimedia applications, with diffusion models showing strong potential for talking-face synthesis. However, most existing works treat speech features as a monolithic representation and fail to capture their fine-grained roles in driving different facial mo...","url_abs":"https://arxiv.org/abs/2509.20128","url_pdf":"https://arxiv.org/pdf/2509.20128v2","authors":"[\"Tianle Lyu\",\"Junchuan Zhao\",\"Ye Wang\"]","published":"2025-09-24T13:54:52Z","proceeding":"cs.GR","tasks":"[\"cs.GR\",\"cs.AI\",\"cs.CV\",\"cs.MM\"]","methods":"[\"Diffusion Model\"]","has_code":false}