OpenCodePapers

video-to-sound-generation-on-vggsound-v2a

Audio GenerationVideo-to-Sound Generation
Dataset Link
Results over time
Click legend items to toggle metrics. Hover points for model names.
Leaderboard
PaperCodeFD (PaSST)KL (PaSST)ImageBind ScoreDe-SyncKL (PANNs)FD (PANNs)FD (VGGish)IS (PANNs)ModelNameReleaseDate
PAVAS: Physics-Aware Video-to-Audio Synthesis✓ Link47.381.3535.410.4461.553.991.15PAVAS-L2025-12-09
Hear What You See: Video-to-Audio Generation with Diffusion Transformer and Semantic-Temporal Alignment-Ranked Direct Preference Optimization55.481.2933.10.411.444.360.9918.41VisioSonic w/ STAR-DPO (44.1kHz)2026-06-03
MMAudio: Taming Multimodal Joint Training for High-Quality Video-to-Audio Synthesis✓ Link60.61.433.220.4421.654.720.9717.4MMAudio-L-44.1kHz (extra data)2024-12-19
MMAudio: Taming Multimodal Joint Training for High-Quality Video-to-Audio Synthesis✓ Link61.881.4132.990.4431.664.741.1317.41MMAudio-M-44.1kHz (extra data)2024-12-19
MMAudio: Taming Multimodal Joint Training for High-Quality Video-to-Audio Synthesis✓ Link65.251.4432.270.4441.675.551.6618.02MMAudio-S-44.1kHz (extra data)2024-12-19
MMAudio: Taming Multimodal Joint Training for High-Quality Video-to-Audio Synthesis✓ Link70.191.5929.130.4831.655.220.7914.44MMAudio-S-16kHz (extra data)2024-12-19
Foley-Flow: Coordinated Video-to-Audio Generation with Masked Audio-Visual Alignment and Dynamic Conditional Flows0.97Foley-Flow2026-03-09
Benchmarking Single-Factor Physical Video-to-Audio Generation34.890.4051.37MMAudio-Phys (w/ Caption) (extra data)2026-05-28
Precise Video-to-Audio Generation with Cross-Modal Alignment in Latent Space✓ Link1.5630.0719.68Flowley + SoundCap2026-07-07