Falcon Mamba: The First Competitive Attention-free 7B Language Model

Benchmark Model Rank Results
common-sense-reasoning-on-arc-challengeFalcon Mamba 7B (25-shot)#14Accuracy: 62.03
multi-task-language-understanding-on-mmluFalcon Mamba 7B (5-shot)#49Average (%): 62.11
common-sense-reasoning-on-winograndeFalcon Mamba 7B#61Accuracy: 73.64
sentence-completion-on-hellaswagFalcon Mamba 7B#67Accuracy: 80.82
multi-task-language-understanding-on-mmlu-proFalcon Mamba 7B (5-shot)#136Accuracy: 14.47
arithmetic-reasoning-on-gsm8kFalcon Mamba 7B (5-shot)#221Accuracy: 52.54