StructBERT: Incorporating Language Structures into Pre-training for Deep Language Understanding

Benchmark Model Rank Results
linguistic-acceptability-on-colaStructBERTRoBERTa ensembleAccuracy: 69.2%
natural-language-inference-on-multinliAdv-RoBERTa ensembleMatched: 91.1Mismatched: 90.7
natural-language-inference-on-qnliStructBERTRoBERTa ensembleAccuracy: 99.2%
natural-language-inference-on-rteAdv-RoBERTa ensembleAccuracy: 88.7%
natural-language-inference-on-wnliStructBERTRoBERTa ensembleAccuracy: 89.7
paraphrase-identification-on-quora-questionStructBERTRoBERTa ensembleF1: 74.4Accuracy: 90.7
semantic-textual-similarity-on-mrpcStructBERTRoBERTa ensembleAccuracy: 91.5%F1: 93.6%
semantic-textual-similarity-on-sts-benchmarkStructBERTRoBERTa ensemblePearson Correlation: 0.928Spearman Correlation: 0.924
sentiment-analysis-on-sst-2-binaryStructBERTRoBERTa ensembleAccuracy: 97.1