{"ID":6023634,"CreatedAt":"2026-07-08T01:00:23.257252134Z","UpdatedAt":"2026-07-10T15:32:35.971326013Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2607.06405","arxiv_id":"2607.06405","title":"Precise Video-to-Audio Generation with Cross-Modal Alignment in Latent Space","abstract":"Video-to-audio (V2A) generation aims to synthesize realistic audio that is both semantically consistent with and temporally synchronized to a silent video. Despite recent progress, many methods still rely on multi-stage training, resulting in high computational costs and long runtimes, or transform visual input into text to leverage pretrained text-to-audio models, sacrificing fine-grained temporal cues. To overcome these limitations, we propose Flowley, an end-to-end, single-stage training architecture that produces soundtracks by combining visual features with textual prompts. Crucially, we introduce Progressive Soft-masked Cross-Attention, which embeds audio-visual synchronization directly within its attention mechanism, adding zero additional computational cost compared to standard attention layers. We further observe that existing V2A benchmarks lack sound-oriented descriptive captions, which can potentially degrade the quality of the synthesized audio. To remedy this, we propose SoundCap, a plug-and-play pipeline for creating detailed, sound-aware captions that guide the model. Remarkably, without integrating any pretrained audio-visual alignment modules, Flowley achieves state-of-the-art performance on VGGSound across multiple metrics. Moreover, by incorporating SoundCap, we further exceed the performance of the strongest existing close-sourced methods in terms of audio quality in the zero-shot setting.","short_abstract":"Video-to-audio (V2A) generation aims to synthesize realistic audio that is both semantically consistent with and temporally synchronized to a silent video. Despite recent progress, many methods still rely on multi-stage training, resulting in high computational costs and long runtimes, or transform visual input into te...","url_abs":"https://arxiv.org/abs/2607.06405","url_pdf":"https://arxiv.org/pdf/2607.06405v1","authors":"[\"Thanh V. T. Tran\",\"Ngoc-Son Nguyen\",\"Luong Tran\",\"Long-Khanh Pham\",\"Paarth Neekhara\",\"Shezheen Hussain\",\"Van Nguyen\"]","published":"2026-07-07T15:34:43Z","proceeding":"cs.MM","tasks":"[\"cs.MM\",\"cs.SD\"]","methods":"[]","has_code":false}