RealFormer: Transformer Likes Residual Attention

Benchmark Model Rank Results
paraphrase-identification-on-quora-question-pairsRealFormer#4F1: 88.28Accuracy: 91.34
semantic-textual-similarity-on-sts-benchmarkRealFormer#13Pearson Correlation: 0.9011Spearman Correlation: 0.8988
natural-language-inference-on-multinliRealFormer#22Matched: 86.28Mismatched: 86.34
semantic-textual-similarity-on-mrpcRealFormer#24Accuracy: 87.01%F1: 90.91%
linguistic-acceptability-on-colaRealFormer#26Accuracy: 59.83%
natural-language-inference-on-qnliRealFormer#26Accuracy: 91.89%
sentiment-analysis-on-sst-2-binary-classificationRealFormer#35Accuracy: 94.04
natural-language-inference-on-rteRealFormer#43Accuracy: 73.7%