Leveraging Unimodal Self-Supervised Learning for Multimodal Audio-Visual Speech Recognition

Benchmark Model Rank Results
audio-visual-speech-recognition-on-lrs2MoCo + wav2vec (w/o extLM)#3Test WER: 2.6
automatic-speech-recognition-on-lrs2MoCo + wav2vec (w/o extLM)#3Test WER: 2.7
lipreading-on-lip-reading-in-the-wildMoCo + Wav2Vec by SJTU LUMIA#12Top-1 Accuracy: 85.0
lipreading-on-lrs2MoCo + wav2vec (w/o extLM)#9Word Error Rate (WER): 43.2