The Falcon Series of Open Language Models

Benchmark Model Rank Results
language-modelling-on-lambadaFalcon-180B (1-shot)#4Accuracy: 84.4
common-sense-reasoning-on-winograndeFalcon-180B (5-shot)#10Accuracy: 87.1
question-answering-on-boolqFalcon-180B#13Accuracy: 87.8
code-generation-on-humanevalFalcon-180B#115Pass@1: 35.4
multi-task-language-understanding-on-mmluFalcon 40B–Average (%): 57.0
multi-task-language-understanding-on-mmluFalcon 7B (5-shot)–Average (%): 28.0
sentence-completion-on-hellaswagFalcon-180B (0-shot)–Accuracy: 85.9
sentence-completion-on-hellaswagFalcon-40B (0-shot)–Accuracy: 82.7
sentence-completion-on-hellaswagFalcon-7B (0-shot)–Accuracy: 76.3