Kimi-VL Technical Report

Benchmark Model Rank Results
zero-shot-video-question-answer-on-egoschema-fullsetKimi-VL-A3B-Instruct#1Accuracy: 78.5
visual-question-answering-on-mm-vetKimi-VL-A3B-Thinking-2506#3GPT-4 score: 78.1
visual-question-answering-on-mm-vetKimi-VL-A3B-Instruct#18GPT-4 score: 66.7
visual-question-answering-vqa-on-infographicvqaKimi-VL-A3B-Instruct#5ANLS: 83.2
video-question-answering-on-tomatoKimi-VL-A3B-Instruct#11Accuracy: 31.7
vision-language-understanding-on-mmbench-v1-1Kimi-VL-A3B-Thinking-2506#14Accuracy: 84.4
vision-language-understanding-on-mmbench-v1-1Kimi-VL-A3B-Instruct#16Accuracy: 83.1
video-question-answering-on-video-mme-with-subtitlesKimi-VL-A3B-Instruct (w/ subs)#20Accuracy: 72.6
video-question-answering-on-video-mme-with-subtitlesKimi-VL-A3B-Thinking-2506 (w/ subs)#23Accuracy: 71.9
optical-character-recognition-ocr-on-ocrbenchKimi-VL-A3B-Thinking-2506#21Score: 869
video-question-answering-on-video-mmmuKimi-VL-A3B-Thinking-2506#21Accuracy: 65.2
video-question-answering-on-video-mmmuKimi-VL-A3B-Instruct#28Accuracy: 52.6
computer-use-agents-on-screenspot-v2Kimi-VL-A3B-Instruct#23Accuracy: 92.8
multimodal-reasoning-on-mathvisionKimi-VL-A3B-Thinking-2506#25Accuracy: 56.9
multimodal-reasoning-on-mathvisionKimi-VL-A3B-Instruct#59Accuracy: 21.4
video-question-answering-on-mlvuKimi-VL-A3B-Instruct#25Accuracy: 74.2
video-question-answering-on-mmvuKimi-VL-A3B-Thinking-2506#26Accuracy: 57.5
video-question-answering-on-mmvuKimi-VL-A3B-Instruct#30Accuracy: 52.2
computer-use-agents-on-screenspot-proKimi-VL-A3B-Thinking-2506#29Accuracy: 52.8
multimodal-reasoning-on-mathvistaKimi-VL-A3B-Thinking-2506#29Accuracy: 80.1
multimodal-reasoning-on-mathvistaKimi-VL-A3B-Instruct#67Accuracy: 68.7
zero-shot-video-question-answer-on-video-mme-w-o-subsKimi-VL-A3B-Instruct#29Accuracy (%): 67.8
visual-question-answering-vqa-on-blinkKimi-VL-A3B-Instruct#30Accuracy: 57.3
vision-language-understanding-on-mmstarKimi-VL-A3B-Thinking-2506#40Accuracy: 70.4
vision-language-understanding-on-mmstarKimi-VL-A3B-Instruct#78Accuracy: 61.3
multimodal-reasoning-on-mmmu-proKimi-VL-A3B-Thinking-2506#43Accuracy: 46.3
multimodal-reasoning-on-mmmuKimi-VL-A3B-Thinking-2506#51Accuracy: 64.0
multimodal-reasoning-on-mmmuKimi-VL-A3B-Instruct#66Accuracy: 57.0
visual-question-answering-vqa-on-realworldqaKimi-VL-A3B-Thinking-2506#52Accuracy: 70.0
visual-question-answering-vqa-on-realworldqaKimi-VL-A3B-Instruct#57Accuracy: 68.1