Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution

Benchmark Model Rank Results
temporal-relation-extraction-on-vinogroundQwen2-VL-72B#1Text Score: 50.4Video Score: 32.6Group Score: 17.4
temporal-relation-extraction-on-vinogroundQwen2-VL-7B#4Text Score: 40.2Video Score: 32.4Group Score: 15.2
video-question-answering-on-implicitqaQwen2 VL - 7B#2Average Accuracy: 44.9Macro Average Accuracy: 46.0
zero-shot-video-question-answer-on-egoschema-fullsetQwen2-VL-72B#2Accuracy: 77.9
zero-shot-video-question-answer-on-egoschema-fullsetQwen2-VL-7B#14Accuracy: 66.7
zero-shot-video-question-answer-on-egoschema-fullsetQwen2-VL-2B#38Accuracy: 54.9
visual-question-answering-on-docvqa-testQwen2-VL-72B#3ANLS: 0.965
visual-question-answering-on-docvqa-testQwen2-VL-7B#15ANLS: 0.945
visual-question-answering-on-docvqa-testQwen2-VL-2B#43ANLS: 0.901
visual-question-answering-vqa-on-mmeQwen2-VL-72B#3Score: 2482.7
visual-question-answering-vqa-on-mmeQwen2-VL-7B#10Score: 2326.8
visual-question-answering-vqa-on-mmeQwen2-VL-2B#20Score: 1872.0
visual-question-answering-vqa-on-vlm2-benchQwen2-VL-7B#3GC-mat: 27.80GC-trk: 19.18OC-cpr: 68.06OC-cnt: 45.99…
visual-question-answering-on-mm-vet-v2Qwen2-VL-72B (qwen-vl-max-0809)#4GPT-4 score: 66.9±0.3Params: 72B
visual-question-answering-vqa-on-mtvqaQwen2-VL-72B#4Accuracy: 30.9
visual-question-answering-vqa-on-mtvqaQwen2-VL-7B#5Accuracy: 25.6
visual-question-answering-vqa-on-mtvqaQwen2-VL-2B#6Accuracy: 18.1
video-question-answering-on-ovbenchQwen2-VL (7B)#5AVG: 49.7
zero-shot-video-question-answer-on-vnbenchQwen2-VL-7B#5Accuracy: 33.9
video-question-answering-on-tvbenchQwen2-VL-72B#6Average Accuracy: 52.7
video-question-answering-on-tvbenchQwen2-VL-7B#13Average Accuracy: 43.8
visual-question-answering-on-mm-vetQwen2-VL-72B#6GPT-4 score: 74.0
visual-question-answering-on-mm-vetQwen2-VL-7B#38GPT-4 score: 62.0
visual-question-answering-on-mm-vetQwen2-VL-2B#70GPT-4 score: 49.5
visual-grounding-on-refcoco-refcoco-refcocog-overallQwen2-VL-72B#8Accuracy: 91.07
visual-question-answering-vqa-on-mmbench-cnQwen2-VL-72B#8Accuracy: 86.6
visual-question-answering-vqa-on-mmbench-cnQwen2-VL-7B#13Accuracy: 80.5
visual-question-answering-vqa-on-mmbench-cnQwen2-VL-2B#18Accuracy: 73.5
video-question-answering-on-next-qaQwen2-VL(7B)#9Accuracy: 81.2
vision-language-understanding-on-mmbench-v1-1Qwen2-VL-72B#11Accuracy: 85.9
vision-language-understanding-on-mmbench-v1-1Qwen2-VL-7B#20Accuracy: 80.7
vision-language-understanding-on-mmbench-v1-1Qwen2-VL-2B#24Accuracy: 72.2
visual-question-answering-vqa-on-mmbench-enQwen2-VL-72B#11Accuracy: 86.5
visual-question-answering-vqa-on-mmbench-enQwen2-VL-7B#21Accuracy: 83.0
visual-question-answering-vqa-on-mmbench-enQwen2-VL-2B#30Accuracy: 74.9
video-question-answering-on-video-mme-with-subtitlesQwen2-VL-72B (w/ subtitles)#15Accuracy: 77.8
optical-character-recognition-ocr-on-ocrbenchQwen2-VL-72B#16Score: 877
visual-question-answering-vqa-on-hallusionbench-average-accuracyQwen2-VL-72B#17Accuracy: 58.1
visual-question-answering-vqa-on-hallusionbench-average-accuracyQwen2-VL-7B#19Accuracy: 50.6
visual-question-answering-vqa-on-hallusionbench-average-accuracyQwen2-VL-2B#22Accuracy: 41.7
visual-question-answering-vqa-on-realworldqaQwen2-VL-72B#17Accuracy: 77.8
visual-question-answering-vqa-on-realworldqaQwen2-VL-7B#53Accuracy: 70.1
visual-question-answering-vqa-on-realworldqaQwen2-VL-2B#81Accuracy: 62.9
chart-question-answering-on-chartqaQwen2-VL-72B#181:1 Accuracy: 88.3
chart-question-answering-on-chartqaQwen2-VL-7B#561:1 Accuracy: 83.0
chart-question-answering-on-chartqaQwen2-VL-2B#891:1 Accuracy: 73.5
video-question-answering-on-mvbenchQwen2-VL-72B#18Avg.: 73.6
video-question-answering-on-mvbenchQwen2-VL-7B#45Avg.: 67.0
video-question-answering-on-mvbenchQwen2-VL-2B#59Avg.: 63.2
zero-shot-video-question-answer-on-video-mme-w-o-subsQwen2-VL-72B#26Accuracy (%): 71.2
zero-shot-video-question-answer-on-video-mme-w-o-subsQwen2-VL-7B#42Accuracy (%): 63.3
zero-shot-video-question-answer-on-video-mme-w-o-subsQwen2-VL-2B#53Accuracy (%): 55.6
natural-language-visual-grounding-on-screenspotQwen2-VL-7B#31Accuracy (%): 42.1
multimodal-reasoning-on-mmmu-proQwen2-VL-72B#45Accuracy: 46.2
multimodal-reasoning-on-mmmu-proQwen2-VL-7B#47Accuracy: 43.5
multimodal-reasoning-on-mmmu-proQwen2-VL-2B#56Accuracy: 37.6
vision-language-understanding-on-mmstarQwen2-VL-72B#51Accuracy: 68.3
vision-language-understanding-on-mmstarQwen2-VL-7B#84Accuracy: 60.7
vision-language-understanding-on-mmstarQwen2-VL-2B#98Accuracy: 48.0
multimodal-reasoning-on-mmmuQwen2-VL-72B#52Accuracy: 64.5
multimodal-reasoning-on-mmmuQwen2-VL-7B#83Accuracy: 54.1
multimodal-reasoning-on-mmmuQwen2-VL-2B#124Accuracy: 41.1
multimodal-reasoning-on-mathvisionQwen2-VL-72B#56Accuracy: 25.9
multimodal-reasoning-on-mathvisionQwen2-VL-7B#62Accuracy: 16.3
multimodal-reasoning-on-mathvisionQwen2-VL-2B#64Accuracy: 12.4
multimodal-reasoning-on-mathvistaQwen2-VL-72B#65Accuracy: 70.5
multimodal-reasoning-on-mathvistaQwen2-VL-7B#101Accuracy: 58.2
multimodal-reasoning-on-mathvistaQwen2-VL-2B#113Accuracy: 43.0