Prune Once for All: Sparse Pre-Trained Language Models

Benchmark Model Rank Results
question-answering-on-squad11-devBERT-Large-uncased-PruneOFA (90% unstruct sparse)#10EM: 83.35F1: 90.2
question-answering-on-squad11-devBERT-Large-uncased-PruneOFA (90% unstruct sparse, QAT Int8)#11EM: 83.22F1: 90.02
question-answering-on-squad11-devBERT-Base-uncased-PruneOFA (85% unstruct sparse)#12EM: 81.1F1: 88.42
question-answering-on-squad11-devBERT-Base-uncased-PruneOFA (85% unstruct sparse, QAT Int8)#13EM: 80.84F1: 88.24
question-answering-on-squad11-devBERT-Base-uncased-PruneOFA (90% unstruct sparse)#14EM: 79.83F1: 87.25
question-answering-on-squad11-devDistilBERT-uncased-PruneOFA (85% unstruct sparse)#19EM: 78.1F1: 85.82
question-answering-on-squad11-devDistilBERT-uncased-PruneOFA (85% unstruct sparse, QAT Int8)#21EM: 77.03F1: 85.13
question-answering-on-squad11-devDistilBERT-uncased-PruneOFA (90% unstruct sparse)#22EM: 76.91F1: 84.82
question-answering-on-squad11-devDistilBERT-uncased-PruneOFA (90% unstruct sparse, QAT Int8)#24EM: 75.62F1: 83.87
natural-language-inference-on-multinli-devBERT-Large-uncased-PruneOFA (90% unstruct sparse)#14Matched: 83.74Mismatched: 84.2
natural-language-inference-on-multinli-devBERT-Large-uncased-PruneOFA (90% unstruct sparse, QAT Int8)#15Matched: 83.47Mismatched: 84.08
natural-language-inference-on-multinli-devBERT-Base-uncased-PruneOFA (85% unstruct sparse)#16Matched: 82.71Mismatched: 83.67
natural-language-inference-on-multinli-devBERT-Base-uncased-PruneOFA (90% unstruct sparse)#19Matched: 81.45Mismatched: 82.43
natural-language-inference-on-multinli-devBERT-Base-uncased-PruneOFA (85% unstruct sparse, QAT Int8)#20Matched: 81.4Mismatched: 82.51
natural-language-inference-on-multinli-devDistilBERT-uncased-PruneOFA (85% unstruct sparse)#21Matched: 81.35Mismatched: 82.03
natural-language-inference-on-multinli-devDistilBERT-uncased-PruneOFA (90% unstruct sparse)#22Matched: 80.68Mismatched: 81.47
natural-language-inference-on-multinli-devDistilBERT-uncased-PruneOFA (85% unstruct sparse, QAT Int8)#23Matched: 80.66Mismatched: 81.14
natural-language-inference-on-multinli-devDistilBERT-uncased-PruneOFA (90% unstruct sparse, QAT Int8)#24Matched: 78.8Mismatched: 80.4