RoBERTa: A Robustly Optimized BERT Pretraining Approach

Benchmark Model Rank Results
common-sense-reasoning-on-commonsenseqaRoBERTa-Large 355M#17Accuracy: 72.1
document-image-classification-on-rvl-cdipRoberta base#25Accuracy: 90.06Parameters: 125M
linguistic-acceptability-on-colaRoBERTa (ensemble)#14Accuracy: 67.8%
natural-language-inference-on-anli-testRoBERTa (Large)#3A1: 72.4A2: 49.8A3: 44.4
natural-language-inference-on-multinliRoBERTa#7Matched: 90.8
natural-language-inference-on-multinliRoBERTa (ensemble)#49Mismatched: 90.2
natural-language-inference-on-qnliRoBERTa (ensemble)#4Accuracy: 98.9%
natural-language-inference-on-rteRoBERTa#15Accuracy: 88.2%
natural-language-inference-on-rteRoBERTa (ensemble)#16Accuracy: 88.2%
natural-language-inference-on-wnliRoBERTa (ensemble)#6Accuracy: 89
question-answering-on-piqaRoBERTa-Large 355M#36Accuracy: 79.4
question-answering-on-quora-question-pairsRoBERTa (ensemble)#5Accuracy: 90.2%
question-answering-on-social-iqaRoBERTa-Large 355M (fine-tuned)#12Accuracy: 76.7
question-answering-on-squad20RoBERTa (single model)#12EM: 86.820F1: 89.795
question-answering-on-squad20-devRoBERTa (no data aug)#3F1: 89.4EM: 86.5
reading-comprehension-on-raceRoBERTa#6Accuracy: 83.2Accuracy (Middle): 86.5Accuracy (High): 81.3
semantic-textual-similarity-on-mrpcRoBERTa (ensemble)#3Accuracy: 92.3%
semantic-textual-similarity-on-sts-benchmarkRoBERTa#5Pearson Correlation: 0.922
sentence-completion-on-hellaswagRoBERTa-Large Ensemble#18Accuracy: 85.5
sentence-completion-on-hellaswagRoBERTa-Large 355M#31Accuracy: 81.7
sentiment-analysis-on-sst-2-binaryRoBERTa (ensemble)#10Accuracy: 96.7
stock-market-prediction-on-astockRoBERTa WWM Ext (News+Factors)#11Accuray: 62.49F1-score: 62.54Recall: 62.51Precision: 62.59
stock-market-prediction-on-astockRoBERTa WWM Ext (News)#12Accuray: 61.34F1-score: 61.48Recall: 61.32Precision: 61.97
task-1-grouping-on-ocwRoBERTa (LARGE)#22# Correct Groups: 29 ± 3Fowlkes Mallows Score (FMS): 26.7 ± .2
text-classification-on-arxiv-10RoBERTa#2Accuracy: 0.779
type-prediction-on-manytypes4typescriptRoBERTa#5Average Accuracy: 59.84Average Precision: 57.45Average Recall: 57.62