Compact Language Models via Pruning and Knowledge Distillation

Benchmark Model Rank Results
common-sense-reasoning-on-winograndeMinitron 8B#37Accuracy: 79.0
common-sense-reasoning-on-winograndeMinitron 4B#60Accuracy: 74.0
sentence-completion-on-hellaswagMinitron 8B#70Accuracy: 80.7
sentence-completion-on-hellaswagMinitron 4B#93Accuracy: 75.0
code-generation-on-humanevalMinitron 8B#119Pass@1: 31.6
code-generation-on-humanevalMinitron 4B#130Pass@1: 23.3
code-generation-on-mbppMinitron 8B#143Accuracy: 35.2
code-generation-on-mbppMinitron 4B#152Accuracy: 28.2
arithmetic-reasoning-on-gsm8kMinitron 8B#235Accuracy: 51.3
arithmetic-reasoning-on-gsm8kMinitron 4B#263Accuracy: 24.1