SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot

Benchmark Model Rank Results
common-sense-reasoning-on-arc-challengeOPT-175B#31Accuracy: 43.94
common-sense-reasoning-on-arc-challengeSparseGPT (175B, 50% Sparsity)#33Accuracy: 41.3
common-sense-reasoning-on-arc-challengeSparseGPT (175B, 4:8 Sparsity)#34Accuracy: 39.85
common-sense-reasoning-on-arc-challengeSparseGPT (175B, 2:4 Sparsity)#35Accuracy: 38.99
common-sense-reasoning-on-arc-challengeOPT-175B (50% Sparsity)#41Accuracy: 25.6
common-sense-reasoning-on-arc-easyOPT-175B#27Accuracy: 71.04
common-sense-reasoning-on-arc-easySparseGPT 175B (50% sparsity)#32Accuracy: 69.65
common-sense-reasoning-on-arc-easySparseGPT (175B, 4:8 Sparsity)#35Accuracy: 68.35
common-sense-reasoning-on-arc-easySparseGPT 175B (2:4 sparsity)#36Accuracy: 67.08
common-sense-reasoning-on-arc-easyOPT 175B (50% Sparsity)#41Accuracy: 28.03
language-modelling-on-lambadaSparseGPT (175B, 2:4 Sparsity)#8Accuracy: 79.47
language-modelling-on-lambadaSparseGPT (175B, 4:8 Sparsity)#9Accuracy: 78.77
language-modelling-on-lambadaSparseGPT (175B, 50% Sparsity)#12Accuracy: 76.51
language-modelling-on-lambadaOPT-175B#14Accuracy: 75.59
language-modelling-on-lambadaOPT-175B (50% Sparsity)#26Accuracy: 0.02
language-modelling-on-wikitext-2SparseGPT (175B, 50% Sparsity)#1Test perplexity: 8.21
language-modelling-on-wikitext-2OPT-175B#2Test perplexity: 8.34
language-modelling-on-wikitext-2SparseGPT (175B, 4:8 Sparsity)#3Test perplexity: 8.45
language-modelling-on-wikitext-2SparseGPT (175B, 2:4 Sparsity)#4Test perplexity: 8.73
language-modelling-on-wikitext-2OPT-175B (50% Sparsity)#37Test perplexity: 234.77
question-answering-on-piqaOPT-175B#27Accuracy: 81.07
question-answering-on-piqaSparseGPT 175B (50% Sparsity)#29Accuracy: 80.63
question-answering-on-piqaSparseGPT 175B (4:8 Sparsity)#34Accuracy: 79.54
question-answering-on-piqaSparseGPT 175B (2:4 Sparsity)#35Accuracy: 79.54
question-answering-on-piqaOPT-175B (50% Sparsity)#61Accuracy: 54.73
question-answering-on-storyclozeOPT-175B#9Accuracy: 79.82
question-answering-on-storyclozeSparseGPT (175B, 50% Sparsity)#10Accuracy: 78.87
question-answering-on-storyclozeSparseGPT (175B, 4:8 Sparsity)#12Accuracy: 77.02
question-answering-on-storyclozeSparseGPT (175B, 2:4 Sparsity)#13Accuracy: 76.19
question-answering-on-storyclozeOPT-175B (50% Sparsity)#15Accuracy: 47.10