OLMo: Accelerating the Science of Language Models

Benchmark Model Rank Results
zero-shot-question-answering-on-openbookqaOLMo 7B#8Accuracy: 50.4
zero-shot-question-answering-on-openbookqaOLMo 1B#13Accuracy: 46.4
question-answering-on-boolqOLMo 7B#61Accuracy: 73.4
question-answering-on-boolqOLMo 1B#97Accuracy: 60.7
common-sense-reasoning-on-arc-easyOLMo 7B#69Accuracy: 65.4
common-sense-reasoning-on-arc-easyOLMo 1B#79Accuracy: 58.1
question-answering-on-piqaOLMo 7B#69Accuracy: 78.4
question-answering-on-piqaOLMo 1B#116Accuracy: 73.7
common-sense-reasoning-on-winograndeOLMo 7B#85Accuracy: 67.9
common-sense-reasoning-on-winograndeOLMo 1B#127Accuracy: 58.9
sentence-completion-on-hellaswagOLMo 7B#86Accuracy: 76.4
sentence-completion-on-hellaswagOLMo 1B#139Accuracy: 62.5
arithmetic-reasoning-on-gsm8kOLMo 7B (8-shot)#262Accuracy: 8.5