| Benchmark | Model | Rank | Results |
|---|---|---|---|
| instruction-following-on-mtbench | BitNet b1.58 2B4T | #16 | Score (1-10): 5.85 |
| question-answering-on-triviaqa | BitNet b1.58 2B4T | #45 | EM: 33.57 |
| multi-task-language-understanding-on-mmlu | BitNet b1.58 2B4T | #61 | Average (%): 53.17 |
| common-sense-reasoning-on-winogrande | BitNet b1.58 2B4T | #70 | Accuracy: 71.90 |
| sentence-completion-on-hellaswag | BitNet b1.58 2B4T | #115 | Accuracy: 68.44 |
| mathematical-reasoning-on-math-500 | BitNet b1.58 2B4T | #126 | Accuracy: 43.40 |
| arithmetic-reasoning-on-gsm8k | BitNet b1.58 2B4T | #208 | Accuracy: 58.38 |