DeBERTa: Decoding-enhanced BERT with Disentangled Attention

Benchmark Model Rank Results
common-sense-reasoning-on-recordDeBERTa-1.5B#2EM: 94.1F1: 94.5
coreference-resolution-on-winograd-schemaDeBERTa-1.5B#4Accuracy: 95.9
linguistic-acceptability-on-cola-devDeBERTa (large)#3Accuracy: 69.5
math-word-problem-solving-on-paramawpsDeBERTa#2Accuracy (%): 74.1
natural-language-inference-on-commitmentbankDeBERTa-1.5B#4Accuracy: 97.2F1: 94.9
natural-language-inference-on-multinliDeBERTa (large)#5Matched: 91.1Mismatched: 91.1
natural-language-inference-on-qnliDeBERTa (large)#8Accuracy: 95.3%
natural-language-inference-on-rteDeBERTa-1.5B#3Accuracy: 93.2%
natural-language-inference-on-wnliDeBERTa#1Accuracy: 94.5
question-answering-on-boolqDeBERTa-1.5B#7Accuracy: 90.4
question-answering-on-copaDeBERTa-Ensemble#4Accuracy: 98.4
question-answering-on-copaDeBERTa-1.5B#5Accuracy: 96.8
question-answering-on-multircDeBERTa-1.5B#3F1: 88.2EM: 63.7
question-answering-on-quora-question-pairsDeBERTa (large)#2Accuracy: 92.3%
question-answering-on-squad20DeBERTalarge#8EM: 88.0F1: 90.7
reading-comprehension-on-raceDeBERTalarge#4Accuracy: 86.8
semantic-textual-similarity-on-sts-benchmarkDeBERTa (large)#57Accuracy: 92.5
sentence-completion-on-hellaswagDeBERTa++#11Accuracy: 93
sentiment-analysis-on-sst-2-binaryDeBERTa (large)#14Accuracy: 96.5
word-sense-disambiguation-on-words-in-contextDeBERTa-Ensemble#4Accuracy: 77.5
word-sense-disambiguation-on-words-in-contextDeBERTa-1.5B#7Accuracy: 76.4