Muppet: Massive Multi-task Representations with Pre-Finetuning

Benchmark Model Rank Results
text-summarization-on-gigawordMUPPET BART Large#2ROUGE-1: 40.4ROUGE-2: 20.54ROUGE-L: 36.21
text-summarization-on-reddit-tifuMUPPET BART Large#2ROUGE-1: 30.3ROUGE-2: 11.25ROUGE-L: 24.92
natural-language-inference-on-rteMUPPET Roberta Large#4Accuracy: 92.8%
sentiment-analysis-on-sst-2-binary-classificationMUPPET Roberta Large#5Accuracy: 97.4
sentiment-analysis-on-sst-2-binary-classificationMUPPET Roberta base#13Accuracy: 96.7
common-sense-reasoning-on-commonsenseqaMUPPET Roberta Large#7Accuracy: 79.2
abstractive-text-summarization-on-cnn-daily-mailMUPPET BART Large#11ROUGE-1: 44.45ROUGE-2: 21.25ROUGE-L: 41.4
question-answering-on-boolqMUPPET Roberta Large#15Accuracy: 87.5
question-answering-on-boolqMUPPET Roberta Base#30Accuracy: 83.8
sentence-completion-on-hellaswagMUPPET Roberta Large#30Accuracy: 86.4