Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Vision-Language Models

Benchmark Model Rank Results
visual-question-answering-vqa-on-countbenchqaMolmo 72B#2Accuracy: 91.2
visual-question-answering-vqa-on-infographicvqaMolmo 72B#6ANLS: 81.9
visual-question-answering-vqa-on-textvqa-valMolmo 72B#11Accuracy: 83.1
visual-question-answering-on-docvqa-testMolmo 72B#19ANLS: 0.935
chart-question-answering-on-chartqaMolmo 72B#241:1 Accuracy: 87.3
visual-question-answering-vqa-on-realworldqaMolmo 72B#24Accuracy: 75.2
multimodal-reasoning-on-mmmuMolmo 72B#82Accuracy: 54.1
multimodal-reasoning-on-mathvistaMolmo 72B#96Accuracy: 58.6