SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot

Benchmark Model Rank Results
language-modelling-on-wikitext-2SparseGPT (175B, 50% Sparsity)#1Test perplexity: 8.21
language-modelling-on-wikitext-2OPT-175B#2Test perplexity: 8.34
language-modelling-on-wikitext-2SparseGPT (175B, 4:8 Sparsity)#3Test perplexity: 8.45
language-modelling-on-wikitext-2SparseGPT (175B, 2:4 Sparsity)#4Test perplexity: 8.73
language-modelling-on-wikitext-2OPT-175B (50% Sparsity)#37Test perplexity: 234.77
language-modelling-on-lambadaSparseGPT (175B, 2:4 Sparsity)#9Accuracy: 79.47
language-modelling-on-lambadaSparseGPT (175B, 4:8 Sparsity)#10Accuracy: 78.77
language-modelling-on-lambadaSparseGPT (175B, 50% Sparsity)#13Accuracy: 76.51
language-modelling-on-lambadaOPT-175B#15Accuracy: 75.59
language-modelling-on-lambadaOPT-175B (50% Sparsity)#66Accuracy: 0.02
question-answering-on-storyclozeOPT-175B#9Accuracy: 79.82
question-answering-on-storyclozeSparseGPT (175B, 50% Sparsity)#10Accuracy: 78.87
question-answering-on-storyclozeSparseGPT (175B, 4:8 Sparsity)#12Accuracy: 77.02
question-answering-on-storyclozeSparseGPT (175B, 2:4 Sparsity)#13Accuracy: 76.19
question-answering-on-storyclozeOPT-175B (50% Sparsity)#15Accuracy: 47.10
common-sense-reasoning-on-arc-challengeOPT-175B#32Accuracy: 43.94
common-sense-reasoning-on-arc-challengeSparseGPT (175B, 50% Sparsity)#34Accuracy: 41.3
common-sense-reasoning-on-arc-challengeSparseGPT (175B, 4:8 Sparsity)#35Accuracy: 39.85
common-sense-reasoning-on-arc-challengeSparseGPT (175B, 2:4 Sparsity)#36Accuracy: 38.99
common-sense-reasoning-on-arc-challengeOPT-175B (50% Sparsity)#42Accuracy: 25.6
question-answering-on-piqaOPT-175B#52Accuracy: 81.07
question-answering-on-piqaSparseGPT 175B (50% Sparsity)#56Accuracy: 80.63
question-answering-on-piqaSparseGPT 175B (4:8 Sparsity)#65Accuracy: 79.54
question-answering-on-piqaSparseGPT 175B (2:4 Sparsity)#66Accuracy: 79.54
question-answering-on-piqaOPT-175B (50% Sparsity)#178Accuracy: 54.73
common-sense-reasoning-on-arc-easyOPT-175B#55Accuracy: 71.04
common-sense-reasoning-on-arc-easySparseGPT 175B (50% sparsity)#61Accuracy: 69.65
common-sense-reasoning-on-arc-easySparseGPT (175B, 4:8 Sparsity)#66Accuracy: 68.35
common-sense-reasoning-on-arc-easySparseGPT 175B (2:4 sparsity)#70Accuracy: 67.08
common-sense-reasoning-on-arc-easyOPT 175B (50% Sparsity)#98Accuracy: 28.03