{"ID":2840525,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_url":"https://arxiv.org/abs/2511.13219","arxiv_id":"2511.13219","title":"FoleyBench: A Benchmark For Video-to-Audio Models","abstract":"Video-to-audio generation (V2A) is of increasing importance in domains such as film post-production, AR/VR, and sound design, particularly for the creation of Foley sound effects synchronized with on-screen actions. Foley requires generating audio that is both semantically aligned with visible events and temporally aligned with their timing. Yet, there is a mismatch between evaluation and downstream applications due to the absence of a benchmark tailored to Foley-style scenarios. We find that 74% of videos from past evaluation datasets have poor audio-visual correspondence. Moreover, they are dominated by speech and music, domains that lie outside the use case for Foley. To address this gap, we introduce FoleyBench, the first large-scale benchmark explicitly designed for Foley-style V2A evaluation. FoleyBench contains 5,000 (video, ground-truth audio, text caption) triplets, each featuring visible sound sources with audio causally tied to on-screen events. The dataset is built using an automated, scalable pipeline applied to in-the-wild internet videos from YouTube-based and Vimeo-based sources. Compared to past datasets, we show that videos from FoleyBench have stronger coverage of sound categories from a taxonomy specifically designed for Foley sound. Each clip is further labeled with metadata capturing source complexity, UCS/AudioSet category, and video length, enabling fine-grained analysis of model performance and failure modes. We benchmark several state-of-the-art V2A models, evaluating them on audio quality, audio-video alignment, temporal synchronization, and audio-text consistency. Samples are available at: https://gclef-cmu.org/foleybench","short_abstract":"Video-to-audio generation (V2A) is of increasing importance in domains such as film post-production, AR/VR, and sound design, particularly for the creation of Foley sound effects synchronized with on-screen actions. Foley requires generating audio that is both semantically aligned with visible events and temporally ali...","url_abs":"https://arxiv.org/abs/2511.13219","url_pdf":"https://arxiv.org/pdf/2511.13219v2","authors":"[\"Satvik Dixit\",\"Koichi Saito\",\"Zhi Zhong\",\"Yuki Mitsufuji\",\"Chris Donahue\"]","published":"2025-11-17T10:34:59Z","proceeding":"cs.SD","tasks":"[\"cs.SD\",\"cs.AI\",\"eess.AS\"]","methods":"[]","project_urls":"[\"https://gclef-cmu.org/foleybench\"]","has_code":false,"code_links":[{"ID":606976,"CreatedAt":"2026-06-01T04:54:23.091178241Z","UpdatedAt":"2026-06-01T04:54:23.091178241Z","DeletedAt":null,"paper_id":2840525,"paper_url":"https://arxiv.org/abs/2511.13219","paper_title":"FoleyBench: A Benchmark For Video-to-Audio Models","repo_url":"https://github.com/hkchengrex/av-benchmark","is_official":false,"mentioned_in_paper":false,"mentioned_in_github":true,"github_stars":0}]}