Qwen3.5: Towards Native Multimodal Agents

Benchmark Model Rank Results
agents-on-bfclv4Qwen3.5-397B-A17B#1Accuracy: 72.9
agents-on-bfclv4Qwen3.5-9B#5Accuracy: 66.1
multi-task-language-understanding-on-c-evalQwen3.5-397B-A17B#1Accuracy: 93.0
multi-task-language-understanding-on-c-evalQwen3.5-122B-A10B#4Accuracy: 91.9
multi-task-language-understanding-on-c-evalQwen3.5-27B#7Accuracy: 90.5
multi-task-language-understanding-on-c-evalQwen3.5-35B-A3B#11Accuracy: 90.2
multi-task-language-understanding-on-c-evalQwen3.5-9B#16Accuracy: 88.2
multi-task-language-understanding-on-mmlu-reduxQwen3.5-397B-A17B#1Accuracy: 94.9
multi-task-language-understanding-on-mmlu-reduxQwen3.5-122B-A10B#2Accuracy: 94.0
multi-task-language-understanding-on-mmlu-reduxQwen3.5-35B-A3B#6Accuracy: 93.3
multi-task-language-understanding-on-mmlu-reduxQwen3.5-27B#8Accuracy: 93.2
multi-task-language-understanding-on-mmlu-reduxQwen3.5-9B#14Accuracy: 91.1
multi-task-language-understanding-on-mmmluQwen3.5-397B-A17B#1Accuracy: 88.5
multi-task-language-understanding-on-mmmluQwen3.5-122B-A10B#4Accuracy: 86.7
multi-task-language-understanding-on-mmmluQwen3.5-27B#5Accuracy: 85.9
multi-task-language-understanding-on-mmmluQwen3.5-35B-A3B#6Accuracy: 85.2
multi-task-language-understanding-on-mmmluQwen3.5-9B#11Accuracy: 81.2
multimodal-reasoning-on-mathvisionQwen3.5-397B-A17B#1Accuracy: 88.6
multimodal-reasoning-on-mathvisionQwen3.5-122B-A10B#4Accuracy: 86.2
multimodal-reasoning-on-mathvisionQwen3.5-27B#5Accuracy: 86.0
multimodal-reasoning-on-mathvisionQwen3.5-35B-A3B#8Accuracy: 83.9
multimodal-reasoning-on-mathvisionQwen3.5-9B#12Accuracy: 78.9
multimodal-reasoning-on-mathvistaQwen3.5-397B-A17B#1Accuracy: 90.3
multimodal-reasoning-on-mathvistaQwen3.5-27B#3Accuracy: 87.8
multimodal-reasoning-on-mathvistaQwen3.5-122B-A10B#4Accuracy: 87.4
multimodal-reasoning-on-mathvistaQwen3.5-35B-A3B#8Accuracy: 86.2
multimodal-reasoning-on-mathvistaQwen3.5-9B#12Accuracy: 85.7
multimodal-reasoning-on-mmmuQwen3.5-397B-A17B#1Accuracy: 85.0
multimodal-reasoning-on-mmmuQwen3.5-122B-A10B#3Accuracy: 83.9
multimodal-reasoning-on-mmmuQwen3.5-27B#6Accuracy: 82.3
multimodal-reasoning-on-mmmuQwen3.5-35B-A3B#8Accuracy: 81.4
multimodal-reasoning-on-mmmuQwen3.5-9B#13Accuracy: 78.4
multimodal-reasoning-on-zerobenchQwen3.5-397B-A17B#1Accuracy: 12
multimodal-reasoning-on-zerobenchQwen3.5-27B#2Accuracy: 10
multimodal-reasoning-on-zerobenchQwen3.5-122B-A10B#4Accuracy: 9
multimodal-reasoning-on-zerobenchQwen3.5-35B-A3B#5Accuracy: 8
multimodal-reasoning-on-zerobenchQwen3.5-9B#7Accuracy: 3.0
optical-character-recognition-ocr-on-ocrbenchQwen3.5-397B-A17B#1Score: 931
video-question-answering-on-mlvuQwen3.5-122B-A10B#1Accuracy: 87.3
video-question-answering-on-mlvuQwen3.5-397B-A17B#2Accuracy: 86.7
video-question-answering-on-mlvuQwen3.5-27B#5Accuracy: 85.9
video-question-answering-on-mlvuQwen3.5-35B-A3B#7Accuracy: 85.6
video-question-answering-on-mlvuQwen3.5-9B#8Accuracy: 84.4
vision-language-understanding-on-mmbench-v1-1Qwen3.5-397B-A17B#1Accuracy: 93.7
vision-language-understanding-on-mmbench-v1-1Qwen3.5-122B-A10B#2Accuracy: 92.8
vision-language-understanding-on-mmbench-v1-1Qwen3.5-27B#4Accuracy: 92.6
vision-language-understanding-on-mmbench-v1-1Qwen3.5-35B-A3B#6Accuracy: 91.5
vision-language-understanding-on-mmbench-v1-1Qwen3.5-9B#7Accuracy: 90.1
visual-question-answering-vqa-on-hallusionbench-average-accuracyQwen3.5-397B-A17B#1Accuracy: 71.4
visual-question-answering-vqa-on-hallusionbench-average-accuracyQwen3.5-27B#2Accuracy: 70.0
visual-question-answering-vqa-on-hallusionbench-average-accuracyQwen3.5-9B#3Accuracy: 69.3
visual-question-answering-vqa-on-hallusionbench-average-accuracyQwen3.5-35B-A3B#4Accuracy: 67.9
visual-question-answering-vqa-on-hallusionbench-average-accuracyQwen3.5-122B-A10B#5Accuracy: 67.6
zero-shot-video-question-answer-on-video-mme-w-o-subsQwen3.5-122B-A10B#1Accuracy (%): 83.9
zero-shot-video-question-answer-on-video-mme-w-o-subsQwen3.5-397B-A17B#2Accuracy (%): 83.7
zero-shot-video-question-answer-on-video-mme-w-o-subsQwen3.5-27B#3Accuracy (%): 82.8
zero-shot-video-question-answer-on-video-mme-w-o-subsQwen3.5-35B-A3B#4Accuracy (%): 82.5
zero-shot-video-question-answer-on-video-mme-w-o-subsQwen3.5-9B#7Accuracy (%): 78.4
instruction-following-on-multichallengeQwen3.5-397B-A17B#2Accuracy: 67.6
instruction-following-on-multichallengeQwen3.5-122B-A10B#4Accuracy: 61.5
instruction-following-on-multichallengeQwen3.5-27B#6Accuracy: 60.8
instruction-following-on-multichallengeQwen3.5-35B-A3B#7Accuracy: 60.0
instruction-following-on-multichallengeQwen3.5-9B#10Accuracy: 54.5
long-context-understanding-on-longbench-v2Qwen3.5-397B-A17B#2Accuracy: 63.2
long-context-understanding-on-longbench-v2Qwen3.5-27B#6Accuracy: 60.6
long-context-understanding-on-longbench-v2Qwen3.5-122B-A10B#7Accuracy: 60.2
long-context-understanding-on-longbench-v2Qwen3.5-35B-A3B#9Accuracy: 59.0
long-context-understanding-on-longbench-v2Qwen3.5-9B#12Accuracy: 55.2
multi-task-language-understanding-on-includeQwen3.5-397B-A17B#2Accuracy: 85.6
multi-task-language-understanding-on-includeQwen3.5-122B-A10B#3Accuracy: 82.8
multi-task-language-understanding-on-includeQwen3.5-27B#4Accuracy: 81.6
multi-task-language-understanding-on-includeQwen3.5-35B-A3B#6Accuracy: 79.7
multi-task-language-understanding-on-includeQwen3.5-9B#9Accuracy: 75.6
multimodal-reasoning-on-babyvisionQwen3.5-397B-A17B#2Accuracy: 52.3
multimodal-reasoning-on-babyvisionQwen3.5-27B#4Accuracy: 44.6
multimodal-reasoning-on-babyvisionQwen3.5-122B-A10B#5Accuracy: 40.2
multimodal-reasoning-on-babyvisionQwen3.5-35B-A3B#7Accuracy: 38.4
multiple-choice-question-answering-mcqa-on-supergpqaQwen3.5-397B-A17B#2Accuracy: 70.4
multiple-choice-question-answering-mcqa-on-supergpqaQwen3.5-122B-A10B#3Accuracy: 67.1
multiple-choice-question-answering-mcqa-on-supergpqaQwen3.5-27B#5Accuracy: 65.6
multiple-choice-question-answering-mcqa-on-supergpqaQwen3.5-35B-A3B#9Accuracy: 63.4
multiple-choice-question-answering-mcqa-on-supergpqaQwen3.5-9B#12Accuracy: 58.2
object-counting-on-countbenchQwen3.5-27B#2Accuracy: 97.8
object-counting-on-countbenchQwen3.5-35B-A3B#3Accuracy: 97.8
object-counting-on-countbenchQwen3.5-397B-A17B#5Accuracy: 97.2
object-counting-on-countbenchQwen3.5-9B#6Accuracy: 97.2
object-counting-on-countbenchQwen3.5-122B-A10B#7Accuracy: 97.0
visual-grounding-on-refcoco-refcoco-refcocog-overallQwen3.5-397B-A17B#2Accuracy: 92.3
visual-grounding-on-refcoco-refcoco-refcocog-overallQwen3.5-122B-A10B#7Accuracy: 91.3
visual-grounding-on-refcoco-refcoco-refcocog-overallQwen3.5-27B#9Accuracy: 90.9
visual-grounding-on-refcoco-refcoco-refcocog-overallQwen3.5-9B#11Accuracy: 89.7
visual-grounding-on-refcoco-refcoco-refcocog-overallQwen3.5-35B-A3B#13Accuracy: 89.2
agents-on-benchQwen3.5-397B-A17B#3Accuracy: 86.7
agents-on-benchQwen3.5-35B-A3B#6Accuracy: 81.2
agents-on-benchQwen3.5-122B-A10B#11Accuracy: 79.5
agents-on-benchQwen3.5-9B#12Accuracy: 79.1
agents-on-benchQwen3.5-27B#13Accuracy: 79.0
multi-task-language-understanding-on-mmlu-proQwen3.5-397B-A17B#3Accuracy: 87.8
multi-task-language-understanding-on-mmlu-proQwen3.5-122B-A10B#7Accuracy: 86.7
multi-task-language-understanding-on-mmlu-proQwen3.5-27B#11Accuracy: 86.1
multi-task-language-understanding-on-mmlu-proQwen3.5-35B-A3B#13Accuracy: 85.3
multi-task-language-understanding-on-mmlu-proQwen3.5-9B#32Accuracy: 82.5
video-question-answering-on-mvbenchQwen3.5-397B-A17B#3Avg.: 77.6
video-question-answering-on-mvbenchQwen3.5-122B-A10B#5Avg.: 76.6
video-question-answering-on-mvbenchQwen3.5-35B-A3B#12Avg.: 74.8
video-question-answering-on-mvbenchQwen3.5-27B#13Avg.: 74.6
video-question-answering-on-mvbenchQwen3.5-9B#16Avg.: 74.4
vision-language-understanding-on-mmstarQwen3.5-397B-A17B#3Accuracy: 83.8
vision-language-understanding-on-mmstarQwen3.5-122B-A10B#4Accuracy: 82.9
vision-language-understanding-on-mmstarQwen3.5-35B-A3B#5Accuracy: 81.9
vision-language-understanding-on-mmstarQwen3.5-27B#7Accuracy: 81.0
vision-language-understanding-on-mmstarQwen3.5-9B#11Accuracy: 79.7
chart-question-answering-on-charxiv-rqQwen3.5-27B#4Accuracy: 79.5
chart-question-answering-on-charxiv-rqQwen3.5-35B-A3B#10Accuracy: 77.5
chart-question-answering-on-charxiv-rqQwen3.5-122B-A10B#11Accuracy: 77.2
chart-question-answering-on-charxiv-rqQwen3.5-9B#12Accuracy: 73.0
code-generation-on-terminal-bench-2-0-terminus-2Qwen3.5-397B-A17B (Terminus-2)#4Accuracy: 52.5
multimodal-reasoning-on-mmmu-proQwen3.5-397B-A17B#4Accuracy: 79.0
multimodal-reasoning-on-mmmu-proQwen3.5-122B-A10B#7Accuracy: 76.9
multimodal-reasoning-on-mmmu-proQwen3.5-35B-A3B#11Accuracy: 75.1
multimodal-reasoning-on-mmmu-proQwen3.5-27B#12Accuracy: 75.0
multimodal-reasoning-on-mmmu-proQwen3.5-9B#19Accuracy: 70.1
video-question-answering-on-video-mme-with-subtitlesQwen3.5-397B-A17B (w/ subtitles)#4Accuracy: 87.5
video-question-answering-on-video-mme-with-subtitlesQwen3.5-122B-A10B (w/ subtitles)#6Accuracy: 87.3
video-question-answering-on-video-mme-with-subtitlesQwen3.5-27B (w/ subtitles)#7Accuracy: 87.0
video-question-answering-on-video-mme-with-subtitlesQwen3.5-35B-A3B (w/ subtitles)#8Accuracy: 86.6
video-question-answering-on-video-mme-with-subtitlesQwen3.5-9B (w/ subtitles)#11Accuracy: 84.5
video-question-answering-on-video-mmmuQwen3.5-397B-A17B#4Accuracy: 84.7
video-question-answering-on-video-mmmuQwen3.5-27B#8Accuracy: 82.3
video-question-answering-on-video-mmmuQwen3.5-122B-A10B#9Accuracy: 82.0
video-question-answering-on-video-mmmuQwen3.5-35B-A3B#10Accuracy: 80.4
video-question-answering-on-video-mmmuQwen3.5-9B#13Accuracy: 78.9
visual-question-answering-vqa-on-realworldqaQwen3.5-122B-A10B#5Accuracy: 85.1
visual-question-answering-vqa-on-realworldqaQwen3.5-35B-A3B#6Accuracy: 84.1
visual-question-answering-vqa-on-realworldqaQwen3.5-397B-A17B#8Accuracy: 83.9
visual-question-answering-vqa-on-realworldqaQwen3.5-27B#9Accuracy: 83.7
visual-question-answering-vqa-on-realworldqaQwen3.5-9B#13Accuracy: 80.3
visual-question-answering-vqa-on-simplevqaQwen3.5-122B-A10B#5Accuracy: 61.7
visual-question-answering-vqa-on-simplevqaQwen3.5-35B-A3B#6Accuracy: 58.3
visual-question-answering-vqa-on-simplevqaQwen3.5-27B#7Accuracy: 56.0
visual-question-answering-vqa-on-simplevqaQwen3.5-9B#9Accuracy: 51.2
zero-shot-object-detection-on-odinw13Qwen3.5-397B-A17B (zero-shot)#5mAP: 47.0
video-question-answering-on-lvbenchQwen3.5-397B-A17B#6Accuracy: 75.5
video-question-answering-on-lvbenchQwen3.5-122B-A10B#7Accuracy: 74.4
video-question-answering-on-lvbenchQwen3.5-27B#8Accuracy: 73.6
video-question-answering-on-lvbenchQwen3.5-35B-A3B#11Accuracy: 71.4
video-question-answering-on-lvbenchQwen3.5-9B#12Accuracy: 70.0
video-question-answering-on-mmvuQwen3.5-397B-A17B#7Accuracy: 75.4
video-question-answering-on-mmvuQwen3.5-122B-A10B#8Accuracy: 74.7
video-question-answering-on-mmvuQwen3.5-27B#9Accuracy: 73.3
video-question-answering-on-mmvuQwen3.5-35B-A3B#10Accuracy: 72.3
video-question-answering-on-mmvuQwen3.5-9B#19Accuracy: 67.8
instruction-following-on-ifbenchQwen3.5-27B#9Accuracy: 76.5
instruction-following-on-ifbenchQwen3.5-397B-A17B#10Accuracy: 76.5
instruction-following-on-ifbenchQwen3.5-122B-A10B#11Accuracy: 76.1
instruction-following-on-ifbenchQwen3.5-35B-A3B#18Accuracy: 70.2
instruction-following-on-ifbenchQwen3.5-9B#26Accuracy: 64.5
optical-character-recognition-ocr-on-omnidocbench-v1-5Qwen3.5-397B-A17B#12Accuracy: 90.8
optical-character-recognition-ocr-on-omnidocbench-v1-5Qwen3.5-122B-A10B#14Accuracy: 89.8
optical-character-recognition-ocr-on-omnidocbench-v1-5Qwen3.5-35B-A3B#15Accuracy: 89.3
optical-character-recognition-ocr-on-omnidocbench-v1-5Qwen3.5-27B#17Accuracy: 88.9
optical-character-recognition-ocr-on-omnidocbench-v1-5Qwen3.5-9B#18Accuracy: 87.7
code-generation-on-livecodebench-v6Qwen3.5-397B-A17B#14Pass@1: 83.6
code-generation-on-livecodebench-v6Qwen3.5-27B#17Pass@1: 80.7
code-generation-on-livecodebench-v6Qwen3.5-122B-A10B#21Pass@1: 78.9
code-generation-on-livecodebench-v6Qwen3.5-35B-A3B#25Pass@1: 74.6
code-generation-on-livecodebench-v6Qwen3.5-9B#38Pass@1: 65.6
computer-use-agents-on-androidworldQwen3.5-35B-A3B#14Success Rate: 71.1
question-answering-on-humanity-s-last-exam-hleQwen3.5-397B-A17B#14Accuracy: 28.7
question-answering-on-humanity-s-last-exam-hleQwen3.5-122B-A10B (w/ CoT)#18Accuracy: 25.3
question-answering-on-humanity-s-last-exam-hleQwen3.5-27B (w/ CoT)#20Accuracy: 24.3
question-answering-on-humanity-s-last-exam-hleQwen3.5-35B-A3B (w/ CoT)#24Accuracy: 22.4
multiple-choice-question-answering-mcqa-on-gpqa-diamondQwen3.5-397B-A17B#17Accuracy: 88.4
multiple-choice-question-answering-mcqa-on-gpqa-diamondQwen3.5-122B-A10B#27Accuracy: 86.6
multiple-choice-question-answering-mcqa-on-gpqa-diamondQwen3.5-27B#35Accuracy: 85.5
multiple-choice-question-answering-mcqa-on-gpqa-diamondQwen3.5-35B-A3B#42Accuracy: 84.2
multiple-choice-question-answering-mcqa-on-gpqa-diamondQwen3.5-9B#51Accuracy: 81.7
code-generation-on-swe-bench-verifiedQwen3.5-397B-A17B (Internal agent scaffold)#20Accuracy: 76.4
code-generation-on-swe-bench-verifiedQwen3.5-27B#32Accuracy: 72.4
code-generation-on-swe-bench-verifiedQwen3.5-122B-A10B#35Accuracy: 72.0
code-generation-on-swe-bench-verifiedQwen3.5-35B-A3B#47Accuracy: 69.2
code-generation-on-swe-bench-multilingualQwen3.5-397B-A17B (Internal agent scaffold)#22Accuracy: 69.3
deep-research-agents-on-browsecompQwen3.5-397B-A17B#30Accuracy: 69.0
deep-research-agents-on-browsecompQwen3.5-122B-A10B#36Accuracy: 63.8
deep-research-agents-on-browsecompQwen3.5-27B#37Accuracy: 61.0