StructBERT: Incorporating Language Structures into Pre-training for Deep Language Understanding

Benchmark Model Rank Results
linguistic-acceptability-on-colaStructBERTRoBERTa ensemble–Accuracy: 69.2%
natural-language-inference-on-multinliAdv-RoBERTa ensemble–Matched: 91.1Mismatched: 90.7
natural-language-inference-on-qnliStructBERTRoBERTa ensemble–Accuracy: 99.2%
natural-language-inference-on-rteAdv-RoBERTa ensemble–Accuracy: 88.7%
natural-language-inference-on-wnliStructBERTRoBERTa ensemble–Accuracy: 89.7
paraphrase-identification-on-quora-question-pairsStructBERTRoBERTa ensemble–F1: 74.4Accuracy: 90.7
semantic-textual-similarity-on-mrpcStructBERTRoBERTa ensemble–Accuracy: 91.5%F1: 93.6%
semantic-textual-similarity-on-sts-benchmarkStructBERTRoBERTa ensemble–Pearson Correlation: 0.928Spearman Correlation: 0.924
sentiment-analysis-on-sst-2-binary-classificationStructBERTRoBERTa ensemble–Accuracy: 97.1