A GPT-2 mapper over dual visual encoders claims 16% training cost and better alignment, but test-time use of true class labels makes the comparison invalid for V2A.
Temporally Aligned Audio for Video with Autoregression
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
We introduce V-AURA, the first autoregressive model to achieve high temporal alignment and relevance in video-to-audio generation. V-AURA uses a high-framerate visual feature extractor and a cross-modal audio-visual feature fusion strategy to capture fine-grained visual motion events and ensure precise temporal alignment. Additionally, we propose VisualSound, a benchmark dataset with high audio-visual relevance. VisualSound is based on VGGSound, a video dataset consisting of in-the-wild samples extracted from YouTube. During the curation, we remove samples where auditory events are not aligned with the visual ones. V-AURA outperforms current state-of-the-art models in temporal alignment and semantic relevance while maintaining comparable audio quality. Code, samples, VisualSound and models are available at https://v-aura.notion.site
citation-role summary
background 1
citation-polarity summary
fields
cs.CV 1years
2025 1verdicts
REJECT 1roles
background 1polarities
background 1representative citing papers
citing papers explorer
-
Efficient Video-to-Audio Generation via Multiple Foundation Models Mapper
A GPT-2 mapper over dual visual encoders claims 16% training cost and better alignment, but test-time use of true class labels makes the comparison invalid for V2A.