Audio-Visual Flamingo: Open Audio-Visual Intelligence for Long and Complex Videos

Benchmark Model Rank Results
audio-understanding-reasoning-on-mmarAVF-Instruct–Accuracy: 60.1
audio-understanding-reasoning-on-mmsuAVF-Instruct–Accuracy: 61.5
audio-visual-question-answering-on-dailyomniAVF-Think–Accuracy: 73.9
audio-visual-question-answering-on-omnibenchAVF-Think–Accuracy: 50.6
audio-visual-question-answering-on-worldsenseAVF-Think–Accuracy: 51.6
speech-recognition-on-librispeech-test-otherAVF-Instruct–Word Error Rate (WER): 3.5
speech-recognition-on-spgispeechAVF-Instruct–Word Error Rate (WER): 2.8
video-question-answering-on-video-mme-with-subtitlesAVF-Instruct–Accuracy: 71.2
zero-shot-video-question-answer-on-video-mme-w-o-subsAVF-Instruct–Accuracy (%): 70.7