2 OLMo 2 Furious

Benchmark Model Rank Results
sentence-completion-on-hellaswagOLMo 2 32B#18Accuracy: 89.7
sentence-completion-on-hellaswagOLMo 2 13B#32Accuracy: 86.4
sentence-completion-on-hellaswagOLMo 2 7B#45Accuracy: 83.8
common-sense-reasoning-on-winograndeOLMo 2 32B#19Accuracy: 83.0
common-sense-reasoning-on-winograndeOLMo 2 13B#25Accuracy: 81.5
common-sense-reasoning-on-winograndeOLMo 2 7B#45Accuracy: 77.2
instruction-following-on-ifevalOLMo 2 7B Instruct#20Prompt-level loose-accuracy: 72.3
instruction-following-on-ifevalOLMo 2 13B Instruct#21Prompt-level loose-accuracy: 82.6
instruction-following-on-ifevalOLMo 2 32B Instruct#22Prompt-level loose-accuracy: 85.6
multi-task-language-understanding-on-mmluOLMo 2 7B#43Average (%): 63.7
math-word-problem-solving-on-mathOLMo 2 32B Instruct (4-shot)#86Accuracy: 49.7
math-word-problem-solving-on-mathOLMo 2 13B Instruct (4-shot)#119Accuracy: 39.2
math-word-problem-solving-on-mathOLMo 2 7B Instruct (4-shot)#132Accuracy: 32.5
arithmetic-reasoning-on-gsm8kOLMo 2 32B Instruct (8-shot)#95Accuracy: 87.6
arithmetic-reasoning-on-gsm8kOLMo 2 13B Instruct (8-shot)#97Accuracy: 87.4
arithmetic-reasoning-on-gsm8kOLMo 2 7B Instruct (8-shot)#110Accuracy: 85.1
multi-task-language-understanding-on-mmlu-proOLMo 2 32B#120Accuracy: 46.9
multi-task-language-understanding-on-mmlu-proOLMo 2 13B#125Accuracy: 35.1
multi-task-language-understanding-on-mmlu-proOLMo 2 7B#129Accuracy: 31.0