data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Language

Benchmark Model Rank Results
paraphrase-identification-on-quora-question-pairsdata2vec#16Accuracy: 92.4
linguistic-acceptability-on-coladata2vec#25Accuracy: 60.3%
speech-recognition-on-librispeech-test-cleanfacebook/data2vec-audio-large-960h#26Word Error Rate (WER): 1.38
speech-recognition-on-librispeech-test-cleanfacebook/data2vec-audio-base-960h#93Word Error Rate (WER): 2.19
natural-language-inference-on-qnlidata2vec#30Accuracy: 91.1%
speech-recognition-on-librispeech-test-otherfacebook/data2vec-audio-large-960h#46Word Error Rate (WER): 3.44
speech-recognition-on-librispeech-test-otherdata2vec#56Word Error Rate (WER): 3.7
speech-recognition-on-librispeech-test-otherfacebook/data2vec-audio-base-960h#113Word Error Rate (WER): 6.43
natural-language-inference-on-rtedata2vec#52Accuracy: 69.9%
speech-recognition-on-voxpopulifacebook/data2vec-audio-large-960h#57Word Error Rate (WER): 12.29
speech-recognition-on-voxpopulifacebook/data2vec-audio-base-960h#65Word Error Rate (WER): 17.85
speech-recognition-on-gigaspeech-cleanedfacebook/data2vec-audio-large-960h#60Word Error Rate (WER): 15.68
speech-recognition-on-gigaspeech-cleanedfacebook/data2vec-audio-base-960h#68Word Error Rate (WER): 21.39
speech-recognition-on-amifacebook/data2vec-audio-large-960h#61Word Error Rate (WER): 33.22
speech-recognition-on-amifacebook/data2vec-audio-base-960h#67Word Error Rate (WER): 41.11
speech-recognition-on-ami-cleanedfacebook/data2vec-audio-large-960h#62Word Error Rate (WER): 28.8
speech-recognition-on-ami-cleanedfacebook/data2vec-audio-base-960h#69Word Error Rate (WER): 37.29
speech-recognition-on-voxpopuli-cleanedfacebook/data2vec-audio-large-960h#62Word Error Rate (WER): 7.13
speech-recognition-on-voxpopuli-cleanedfacebook/data2vec-audio-base-960h#67Word Error Rate (WER): 12.76
speech-recognition-on-gigaspeech-testfacebook/data2vec-audio-large-960h#63Word Error Rate (WER): 16.01
speech-recognition-on-gigaspeech-testfacebook/data2vec-audio-base-960h#71Word Error Rate (WER): 21.7
speech-recognition-on-earnings-22facebook/data2vec-audio-large-960h#70Word Error Rate (WER): 26.97
speech-recognition-on-earnings-22facebook/data2vec-audio-base-960h#76Word Error Rate (WER): 40.89
speech-recognition-on-spgispeechfacebook/data2vec-audio-large-960h#72Word Error Rate (WER): 8.35
speech-recognition-on-spgispeechfacebook/data2vec-audio-base-960h#80Word Error Rate (WER): 16.28
image-classification-on-imagenetdata2vec (ViT-H)#141Top 1 Accuracy: 86.6%Number of params: 656M