Qwen2.5-VL Technical Report

Benchmark Model Rank Results
visual-question-answering-vqa-on-vlm2-benchQwen2.5-VL-7B#1GC-mat: 35.91GC-trk: 43.38OC-cpr: 71.39OC-cnt: 41.72…
visual-question-answering-vqa-on-infographicvqaQwen2.5-VL-72B#2ANLS: 87.3
code-generation-on-multipl-eQwen2.5-VL-72B#3Accuracy: 79.5
visual-question-answering-vqa-on-ai2d-test-with-maskQwen2.5-VL-72B#3Accuracy: 88.7
visual-question-answering-vqa-on-ai2d-test-with-maskQwen2.5-VL-7B#5Accuracy: 83.9
visual-question-answering-vqa-on-mtvqaQwen2.5-VL-72B#3Accuracy: 31.7
video-question-answering-on-implicitqaQwen 2.5 VL - 7B#4Average Accuracy: 42.8Macro Average Accuracy: 46.1
video-question-answering-on-tempcompass-overallQwen2.5-VL-72B#4Accuracy: 74.8
zero-shot-video-question-answer-on-egoschema-fullsetQwen2.5-VL-72B#4Accuracy: 76.2
chart-question-answering-on-charxiv-dqQwen2.5-VL-72B#5Accuracy: 87.4
optical-character-recognition-ocr-on-ocrbench-v2Qwen2.5-VL-72B#5English Score: 61.5Chinese Score: 63.7
optical-character-recognition-ocr-on-ocrbench-v2Qwen2.5-VL-7B#7Chinese Score: 57.2
visual-question-answering-on-docvqa-testQwen2.5-VL-72B#5ANLS: 0.964
visual-question-answering-vqa-on-mmeQwen2.5-VL-72B#5Score: 2448
visual-question-answering-vqa-on-mmeQwen2.5-VL-7B#7Score: 2347
optical-character-recognition-ocr-on-omnidocbench-v1-0Qwen2.5-VL-72B#6Edit Distance: 0.226
visual-question-answering-vqa-on-textvqa-valQwen2.5-VL-7B#6Accuracy: 84.9
visual-question-answering-vqa-on-textvqa-valQwen2.5-VL-72B#10Accuracy: 83.5
visual-question-answering-vqa-on-mmbench-cnQwen2.5-VL-72B#7Accuracy: 87.9
zero-shot-video-question-answer-on-longvideobenchQwen2.5-VL-72B#7Accuracy (% ): 60.7
vision-language-understanding-on-mmbench-v1-1Qwen2.5-VL-72B#9Accuracy: 88.4
vision-language-understanding-on-mmbench-v1-1Qwen2.5-VL-7B#17Accuracy: 82.6
visual-question-answering-vqa-on-mmbench-enQwen2.5-VL-72B#10Accuracy: 88.6
visual-question-answering-vqa-on-mmbench-enQwen2.5-VL-7B#20Accuracy: 83.5
optical-character-recognition-ocr-on-ocrbenchQwen2.5-VL-72B#11Score: 885
object-counting-on-countbenchQwen2.5-VL-72B#12Accuracy: 93.6
chart-question-answering-on-chartqaQwen2.5-VL-72B#131:1 Accuracy: 89.5
chart-question-answering-on-chartqaQwen2.5-VL-7B#261:1 Accuracy: 87.3
video-question-answering-on-video-mme-with-subtitlesQwen2.5-VL-72B (w/ subs)#13Accuracy: 79.1
video-question-answering-on-video-mme-with-subtitlesQwen2.5-VL-7B#24Accuracy: 71.6
math-word-problem-solving-on-mathQwen2.5-VL-72B#15Accuracy: 83.0
zero-shot-video-question-answer-on-video-mme-w-o-subsQwen2.5-VL-72B#15Accuracy (%): 73.3
arithmetic-reasoning-on-gsm8kQwen2.5-VL-72B#21Accuracy: 95.3
visual-question-answering-vqa-on-blinkQwen2.5-VL-72B#21Accuracy: 64.4
video-question-answering-on-video-mmmuQwen2.5-VL-72B#22Accuracy: 60.2
video-question-answering-on-video-mmmuQwen2.5-VL-7B#31Accuracy: 47.4
visual-question-answering-vqa-on-realworldqaQwen2.5-VL-72B#23Accuracy: 75.7
visual-question-answering-vqa-on-realworldqaQwen2.5-VL-7B#57Accuracy: 68.5
multi-task-language-understanding-on-mmlu-reduxQwen2.5-VL-72B#24Accuracy: 85.9
chart-question-answering-on-charxiv-rqQwen2.5-VL-72B#25Accuracy: 49.7
video-question-answering-on-mmvuQwen2.5-VL-72B#25Accuracy: 62.9
video-question-answering-on-mlvuQwen2.5-VL-72B#27Accuracy: 74.6
video-question-answering-on-mlvuQwen2.5-VL-7B#39Accuracy: 70.2
computer-use-agents-on-androidworldQwen2.5-VL-72B#29Success Rate: 35
video-question-answering-on-mvbenchQwen2.5-VL-72B#29Avg.: 70.4
video-question-answering-on-mvbenchQwen2.5-VL-7B#35Avg.: 69.6
video-question-answering-on-mvbenchQwen2.5-VL-3B#46Avg.: 67.0
multimodal-reasoning-on-mmmuQwen2.5-VL-72B#35Accuracy: 70.2
code-generation-on-humanevalQwen2.5-VL-72B#36Pass@1: 87.8
vision-language-understanding-on-mmstarQwen2.5-VL-72B#41Accuracy: 70.8
vision-language-understanding-on-mmstarQwen2.5-VL-7B#69Accuracy: 63.9
multimodal-reasoning-on-mathvisionQwen2.5-VL-72B#42Accuracy: 38.1
multimodal-reasoning-on-mathvisionQwen2.5-VL-7B#59Accuracy: 25.1
multimodal-reasoning-on-mmmu-proQwen2.5-VL-72B#42Accuracy: 51.1
multimodal-reasoning-on-mmmu-proQwen2.5-VL-7B#55Accuracy: 38.3
video-question-answering-on-lvbenchQwen2.5-VL-72B#42Accuracy: 47.3
video-question-answering-on-lvbenchQwen2.5-VL-7B#45Accuracy: 45.3
video-question-answering-on-lvbenchQwen2.5-VL-3B#48Accuracy: 43.3
multimodal-reasoning-on-mathvistaQwen2.5-VL-72B#47Accuracy: 74.8
multimodal-reasoning-on-mathvistaQwen2.5-VL-7B#72Accuracy: 68.2
multi-task-language-understanding-on-mmlu-proQwen2.5-VL-72B#72Accuracy: 71.2