| Benchmark | Model | Rank | Results |
|---|---|---|---|
| multi-task-language-understanding-on-mmlu-redux | Mellum 2 Thinking | #23 | Accuracy: 86.2 |
| code-generation-on-livecodebench-v6 | Mellum 2 Thinking | #24 | Pass@1: 75.1 |
| code-generation-on-mbpp | Mellum 2 (base) | #80 | Accuracy: 62.4 |
| multi-task-language-understanding-on-mmlu-pro | Mellum 2 (base) | #97 | Accuracy: 59.3 |
| multiple-choice-question-answering-mcqa-on-gpqa-diamond | Mellum 2 Thinking | #100 | Accuracy: 57.6 |
| code-generation-on-humaneval | Mellum 2 (base) | #107 | Pass@1: 41.5 |
| arithmetic-reasoning-on-gsm8k | Mellum 2 (base) | #144 | Accuracy: 81.7 |