Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning

Benchmark Model Rank Results
question-answering-on-piqaOpen-LLaMA-3B-v2#103Accuracy: 76.2
question-answering-on-piqaSheared-LLaMA-2.7B#109Accuracy: 75.8
question-answering-on-piqaSheared-LLaMA-1.3B#129Accuracy: 73.4
sentence-completion-on-hellaswagSheared-LLaMA-2.7B (50B)#106Accuracy: 70.8
sentence-completion-on-hellaswagOpen-LLaMA-3B-v2#119Accuracy: 67.6
sentence-completion-on-hellaswagSheared-LLaMA-1.3B (50B)#146Accuracy: 60.7