Big Bird: Transformers for Longer Sequences

Benchmark Model Rank Results
document-summarization-on-cnn-daily-mailBigBird-Pegasus#8ROUGE-1: 43.84ROUGE-2: 21.11ROUGE-L: 40.74
linguistic-acceptability-on-colaBigBird#24Accuracy: 58.5%
natural-language-inference-on-multinliBigBird#17Matched: 87.5
natural-language-inference-on-qnliBigBird#21Accuracy: 92.2%
natural-language-inference-on-rteBigBird#38Accuracy: 75.0%
question-answering-on-hotpotqaBigBird-etc#2JOINT-F1: 0.736ANS-F1: 0.755SUP-F1: 0.891
question-answering-on-quora-question-pairsBigBird#14Accuracy: 88.6%
question-answering-on-triviaqaBigBird-etc#30F1: 80.9
question-answering-on-wikihopBigBird-etc#1Test: 82.3
semantic-textual-similarity-on-mrpcBigBird#31F1: 91.5
semantic-textual-similarity-on-sts-benchmarkBigBird#33Spearman Correlation: .878
sentiment-analysis-on-sst-2-binaryBigBird#31Accuracy: 94.6
text-classification-on-yelp-5BigBird#3Accuracy: 72.16%
text-summarization-on-pubmed-1BigBird-Pegasus#11ROUGE-1: 46.32ROUGE-2: 20.65ROUGE-L: 42.33