RealFormer: Transformer Likes Residual Attention

Benchmark Model Rank Results
linguistic-acceptability-on-colaRealFormer#23Accuracy: 59.83%
natural-language-inference-on-multinliRealFormer#21Matched: 86.28Mismatched: 86.34
natural-language-inference-on-qnliRealFormer#22Accuracy: 91.89%
natural-language-inference-on-rteRealFormer#40Accuracy: 73.7%
paraphrase-identification-on-quora-questionRealFormer#4F1: 88.28Accuracy: 91.34
semantic-textual-similarity-on-mrpcRealFormer#24Accuracy: 87.01%F1: 90.91%
semantic-textual-similarity-on-sts-benchmarkRealFormer#13Pearson Correlation: 0.9011Spearman Correlation: 0.8988
sentiment-analysis-on-sst-2-binaryRealFormer#33Accuracy: 94.04