Falcon2-11B Technical Report

Benchmark Model Rank Results
arithmetic-reasoning-on-gsm8kFalcon2-11B (5-shot)–Accuracy: 53.83
code-generation-on-humanevalFalcon2-11B–Pass@1: 29.6
common-sense-reasoning-on-winograndeFalcon2-11B (5-shot)–Accuracy: 78.30
multi-task-language-understanding-on-mmluFalcon2-11B (5-shot)–Average (%): 58.37
multimodal-reasoning-on-mathvistaFalcon2-11B VLM–Accuracy: 37.2
sentence-completion-on-hellaswagFalcon2-11B (10-shot)–Accuracy: 82.91
visual-question-answering-on-docvqa-testFalcon2-11B VLM–ANLS: 0.769
visual-question-answering-vqa-on-mmeFalcon2-11B VLM–Score: 1932
visual-question-answering-vqa-on-realworldqaFalcon2-11B VLM–Accuracy: 61.7
visual-question-answering-vqa-on-seed-seedbenchFalcon2-11B VLM–Accuracy: 72.3