MPNet: Masked and Permuted Pre-training for Language Understanding

Benchmark Model Rank Results
linguistic-acceptability-on-cola-glue-devMPNet Large#3Matthews Correlation: 69.1
natural-language-inference-on-rte-glue-devMPNet Large#3Accuracy: 88.0
paraphrase-identification-on-mrpc-glue-devMPNet Large#3F1: 91.4
semantic-textual-similarity-on-sts-b-glue-devMPNet Large#4Pearson/Spearman Correlation: 92.4
natural-language-inference-on-multinli-devMPNet Large#5Matched: 90.5
natural-language-inference-on-qnli-glue-devMPNet Large#5Accuracy: 94.9
natural-language-understanding-on-glue-devMPNet Large#6GLUE (dev): 89.4
natural-language-understanding-on-glue-testMPNet Base#6GLUE (test): 86.5
reading-comprehension-on-raceMPNet Base#11Accuracy: 76.1
natural-language-inference-on-multinliMPNet Base#12Matched: 88.5
task-1-grouping-on-ocwall-mpnet (BASE)#16Wasserstein Distance (WD): 86.3 ± .4# Correct Groups: 50 ± 4…
linguistic-acceptability-on-colaMPNet Base#21Accuracy: 64.0%
natural-language-inference-on-qnliMPNet Base#21Accuracy: 93.1%
sentiment-analysis-on-sst-2-binary-classificationMPNet Base#21Accuracy: 96.0
question-answering-on-squad20MPNet Base#29F1: 85.8
natural-language-inference-on-rteMPNet Base#31Accuracy: 81.0%
paraphrase-identification-on-quora-question-pairsMPNet Base#34Accuracy: 89.9
semantic-textual-similarity-on-mrpcMPNet Base#37F1: 89.1
question-answering-on-squad11-devMPNet Base#54F1: 92.7