Molmo2: Open Weights and Data for Vision-Language Models with Video Understanding and Grounding

Benchmark Model Rank Results
zero-shot-video-question-answer-on-longvideobenchMolmo2-4B#1Accuracy (% ): 68.0
video-question-answering-on-perception-test-testMolmo2-8B#2Accuracy: 82.1
video-question-answering-on-tomatoMolmo2-4B#2Accuracy: 39.8
visual-question-answering-vqa-on-textvqa-valMolmo2-8B#3Accuracy: 85.7
video-question-answering-on-tempcompass-mcqMolmo2-8B#5Accuracy: 73.4
video-question-answering-on-mvbenchMolmo2-8B#7Avg.: 75.9
visual-question-answering-vqa-on-infographicvqaMolmo2-8B#8ANLS: 80.1
object-counting-on-countbenchMolmo2-O-7B#9Accuracy: 95.1
visual-question-answering-vqa-on-realworldqaMolmo2-8B#18Accuracy: 77.6
zero-shot-video-question-answer-on-egoschema-fullsetMolmo2-8B#22Accuracy: 62.0
visual-question-answering-on-docvqa-testMolmo2-8B#23ANLS: 0.932
video-question-answering-on-lvbenchMolmo2-4B#26Accuracy: 53.9
zero-shot-video-question-answer-on-video-mme-w-o-subsMolmo2-8B#27Accuracy (%): 69.9
visual-question-answering-vqa-on-blinkMolmo2-4B#29Accuracy: 57.5
chart-question-answering-on-chartqaMolmo2-4B#381:1 Accuracy: 86.1
multimodal-reasoning-on-mmmuMolmo2-8B#86Accuracy: 53.0
multimodal-reasoning-on-mathvistaMolmo2-8B#95Accuracy: 58.9