MMAudio: Taming Multimodal Joint Training for High-Quality Video-to-Audio Synthesis

Benchmark Model Rank Results
video-to-sound-generation-on-vgg-soundMMAudio-S-16kHz#1FAD: 0.79FD: 5.22
video-to-sound-generation-on-vgg-soundMMAudio-L-44.1kHz#3FAD: 0.97FD: 4.72
video-to-sound-generation-on-vggsound-v2aMMAudio-L-44.1kHz (extra data)#2FD (PaSST): 60.6KL (PaSST): 1.4ImageBind Score: 33.22…
video-to-sound-generation-on-vggsound-v2aMMAudio-M-44.1kHz (extra data)#3FD (PaSST): 61.88KL (PaSST): 1.41ImageBind Score: 32.99…
video-to-sound-generation-on-vggsound-v2aMMAudio-S-44.1kHz (extra data)#4FD (PaSST): 65.25KL (PaSST): 1.44ImageBind Score: 32.27…
video-to-sound-generation-on-vggsound-v2aMMAudio-S-16kHz (extra data)#5FD (PaSST): 70.19KL (PaSST): 1.59ImageBind Score: 29.13…