data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Language

Benchmark Model Rank Results
image-classification-on-imagenetdata2vec (ViT-H)#129Top 1 Accuracy: 86.6%Number of params: 656M
linguistic-acceptability-on-coladata2vec#22Accuracy: 60.3%
natural-language-inference-on-qnlidata2vec#25Accuracy: 91.1%
natural-language-inference-on-rtedata2vec#48Accuracy: 69.9%
paraphrase-identification-on-quora-questiondata2vec#15Accuracy: 92.4
speech-recognition-on-amifacebook/data2vec-audio-large-960h#61Word Error Rate (WER): 33.22
speech-recognition-on-amifacebook/data2vec-audio-base-960h#67Word Error Rate (WER): 41.11
speech-recognition-on-ami-cleanedfacebook/data2vec-audio-large-960h#62Word Error Rate (WER): 28.8
speech-recognition-on-ami-cleanedfacebook/data2vec-audio-base-960h#69Word Error Rate (WER): 37.29
speech-recognition-on-earnings-22facebook/data2vec-audio-large-960h#62Word Error Rate (WER): 26.97
speech-recognition-on-earnings-22facebook/data2vec-audio-base-960h#68Word Error Rate (WER): 40.89
speech-recognition-on-gigaspeech-cleanedfacebook/data2vec-audio-large-960h#60Word Error Rate (WER): 15.68
speech-recognition-on-gigaspeech-cleanedfacebook/data2vec-audio-base-960h#68Word Error Rate (WER): 21.39
speech-recognition-on-gigaspeech-testfacebook/data2vec-audio-large-960h#63Word Error Rate (WER): 16.01
speech-recognition-on-gigaspeech-testfacebook/data2vec-audio-base-960h#71Word Error Rate (WER): 21.7
speech-recognition-on-librispeech-test-cleanfacebook/data2vec-audio-large-960h#25Word Error Rate (WER): 1.38
speech-recognition-on-librispeech-test-cleanfacebook/data2vec-audio-base-960h#84Word Error Rate (WER): 2.19
speech-recognition-on-librispeech-test-otherfacebook/data2vec-audio-large-960h#39Word Error Rate (WER): 3.44
speech-recognition-on-librispeech-test-otherdata2vec#48Word Error Rate (WER): 3.7
speech-recognition-on-librispeech-test-otherfacebook/data2vec-audio-base-960h#95Word Error Rate (WER): 6.43
speech-recognition-on-spgispeechfacebook/data2vec-audio-large-960h#61Word Error Rate (WER): 8.35
speech-recognition-on-spgispeechfacebook/data2vec-audio-base-960h#69Word Error Rate (WER): 16.28
speech-recognition-on-voxpopulifacebook/data2vec-audio-large-960h#57Word Error Rate (WER): 12.29
speech-recognition-on-voxpopulifacebook/data2vec-audio-base-960h#65Word Error Rate (WER): 17.85
speech-recognition-on-voxpopuli-cleanedfacebook/data2vec-audio-large-960h#62Word Error Rate (WER): 7.13
speech-recognition-on-voxpopuli-cleanedfacebook/data2vec-audio-base-960h#67Word Error Rate (WER): 12.76