Big Bird: Transformers for Longer Sequences

Benchmark Model Rank Results
question-answering-on-wikihopBigBird-etc#1Test: 82.3
question-answering-on-hotpotqaBigBird-etc#2JOINT-F1: 0.736ANS-F1: 0.755SUP-F1: 0.891
text-classification-on-yelp-5BigBird#3Accuracy: 72.16%
document-summarization-on-cnn-daily-mailBigBird-Pegasus#8ROUGE-1: 43.84ROUGE-2: 21.11ROUGE-L: 40.74
text-summarization-on-arxivBIGBIRD-Pegasus#10ROUGE-1: 46.63ROUGE-2: 19.02ROUGE-L: 41.77
text-summarization-on-arxivBIGBIRD-RoBERTa#21ROUGE-1: 41.22ROUGE-2: 16.43ROUGE-L: 36.96
text-summarization-on-pubmedBigBird-Pegasus#12ROUGE-1: 46.32ROUGE-2: 20.65ROUGE-L: 42.33
question-answering-on-quora-question-pairsBigBird#14Accuracy: 88.6%
natural-language-inference-on-multinliBigBird#18Matched: 87.5
natural-language-inference-on-qnliBigBird#25Accuracy: 92.2%
linguistic-acceptability-on-colaBigBird#27Accuracy: 58.5%
semantic-textual-similarity-on-mrpcBigBird#31F1: 91.5
semantic-textual-similarity-on-sts-benchmarkBigBird#33Spearman Correlation: .878
sentiment-analysis-on-sst-2-binary-classificationBigBird#33Accuracy: 94.6
question-answering-on-triviaqaBigBird-etc#38F1: 80.9
natural-language-inference-on-rteBigBird#41Accuracy: 75.0%