DeBERTa: Decoding-enhanced BERT with Disentangled Attention

Benchmark Model Rank Results
linguistic-acceptability-on-cola-glue-devDeBERTa Large#1Matthews Correlation: 70.5
natural-language-inference-on-qnli-glue-devDeBERTa Large#1Accuracy: 95.3
natural-language-inference-on-rte-glue-devDeBERTa Large#1Accuracy: 88.3
natural-language-inference-on-wnliDeBERTa#1Accuracy: 94.5
semantic-textual-similarity-on-sts-b-glue-devDeBERTa Large#1Pearson/Spearman Correlation: 92.8
common-sense-reasoning-on-recordDeBERTa-1.5B#2EM: 94.1F1: 94.5
math-word-problem-solving-on-paramawpsDeBERTa#2Accuracy (%): 74.1
natural-language-inference-on-multinli-devDeBERTa Large#2Matched: 91.1
question-answering-on-quora-question-pairsDeBERTa (large)#2Accuracy: 92.3%
question-answering-on-squad20-devDeBERTa Large#2F1: 90.7
linguistic-acceptability-on-cola-devDeBERTa (large)#3Accuracy: 69.5
natural-language-inference-on-rteDeBERTa-1.5B#3Accuracy: 93.2%
natural-language-understanding-on-glue-devDeBERTa Large#3GLUE (dev): 90.00
question-answering-on-multircDeBERTa-1.5B#3F1: 88.2EM: 63.7
coreference-resolution-on-winograd-schema-challengeDeBERTa-1.5B#4Accuracy: 95.9
natural-language-inference-on-commitmentbankDeBERTa-1.5B#4Accuracy: 97.2F1: 94.9
question-answering-on-copaDeBERTa-Ensemble#4Accuracy: 98.4
question-answering-on-copaDeBERTa-1.5B#5Accuracy: 96.8
word-sense-disambiguation-on-words-in-contextDeBERTa-Ensemble#4Accuracy: 77.5
word-sense-disambiguation-on-words-in-contextDeBERTa-1.5B#7Accuracy: 76.4
natural-language-inference-on-multinliDeBERTa (large)#5Matched: 91.1Mismatched: 91.1
question-answering-on-boolqDeBERTa-1.5B#7Accuracy: 90.4
reading-comprehension-on-raceDeBERTalarge#7Accuracy: 86.8
question-answering-on-squad20DeBERTalarge#8EM: 88.0F1: 90.7
natural-language-inference-on-qnliDeBERTa (large)#10Accuracy: 95.3%
sentence-completion-on-hellaswagDeBERTa++#13Accuracy: 93
sentiment-analysis-on-sst-2-binary-classificationDeBERTa (large)#15Accuracy: 96.5
question-answering-on-squad11-devDeBERTa Large#48F1: 95.5
semantic-textual-similarity-on-sts-benchmarkDeBERTa (large)#57Accuracy: 92.5