From Pixels to Words -- Towards Native One-Vision Models at Scale

Benchmark Model Rank Results
visual-question-answering-vqa-on-seed-seedbenchNEO-ov 9B#9Accuracy: 76.6
video-question-answering-on-videommmuNEO-ov (8B)#11Accuracy: 51.6
visual-question-answering-vqa-on-mmbench-enNEO-ov 9B#14Accuracy: 85.1
visual-question-answering-vqa-on-hallusionbench-average-accuracyNEO-ov 9B#15Accuracy: 59.8
visual-question-answering-vqa-on-blinkNEO-ov 9B#24Accuracy: 62.8
video-question-answering-on-mvbenchNEO-ov 9B#26Avg.: 70.7
video-question-answering-on-video-mmmuNEO-ov 9B#30Accuracy: 51.6
visual-question-answering-on-docvqa-testNEO-ov 9B#31ANLS: 0.919
zero-shot-video-question-answer-on-video-mme-w-o-subsNEO-ov 9B#32Accuracy (%): 67.4
chart-question-answering-on-chartqaNEO-ov 9B#361:1 Accuracy: 86.2
video-question-answering-on-mlvuNEO-ov 9B#38Accuracy: 69.3
multimodal-reasoning-on-mmmuNEO-ov 9B#40Accuracy: 68.1
video-question-answering-on-lvbenchNEO-ov 9B#42Accuracy: 46.4
optical-character-recognition-ocr-on-ocrbenchNEO-ov 9B#49Score: 816
vision-language-understanding-on-mmstarNEO-ov 9B#51Accuracy: 67.3
visual-question-answering-vqa-on-realworldqaNEO-ov 9B#60Accuracy: 67.8