Whisper-Flamingo: Integrating Visual Features into Whisper for Audio-Visual Speech Recognition and Translation

Benchmark Model Rank Results
audio-visual-speech-recognition-on-lrs2Whisper-Flamingo#1Test WER: 1.4
audio-visual-speech-recognition-on-lrs3-tedWhisper-Flamingo#2Word Error Rate (WER): 0.76
automatic-speech-recognition-on-lrs2Whisper#1Test WER: 1.3
speech-recognition-on-lrs3-tedWhisper#1Word Error Rate (WER): 0.68