OpenCodePapers

multimodal-reasoning-on-mmmu-pro

Visual ReasoningMultimodal Reasoning
Dataset Link
Results over time
Click legend items to toggle metrics. Hover points for model names.
Leaderboard
Click a sortable column header to sort.
PaperCodeAccuracyModelNameReleaseDate
Introducing Gemini 3 Flash81.2Google Gemini 3 Flash (Thinking)2025-12-17
Introducing GPT-5.481.2GPT-5.42026-03-05
A new era of intelligence with Gemini 381Gemini 3 Pro2025-11-18
Gemini 3.1 Pro: A smarter model for your most complex tasks80.5Gemini 3.1 Pro2026-02-19
Intern-S2-Preview: Scientific Agentic Foundation Model✓ Link80.46Intern-S2-Preview-397B2026-08-13
Introducing GPT-5.279.5GPT-5.22025-12-11
Kimi K2.6✓ Link79.4Kimi K2.62026-04-20
dots3-note Preview: A Small but Mighty Step Toward Long-Horizon Agency in Real Life✓ Link79.08dots3-note Preview2026-08-13
Qwen3.5: Towards Native Multimodal Agents✓ Link79.0Qwen3.5-397B-A17B2026-02-16
Kimi K2.5: Visual Agentic Intelligence✓ Link78.5Kimi K2.52026-02-02
MiniMax M3: Frontier Coding, 1M Context, Native Multimodality — All in One Model✓ Link78.1MiniMax M32026-05-31
Qwen3.5: Towards Native Multimodal Agents✓ Link76.9Qwen3.5-122B-A10B2026-02-16
Gemma 4 Technical Report✓ Link76.9Gemma 4 31B2026-07-02
Group Entropy-Controlled Policy Optimization76.6GEPO (Qwen3.5-9B)2026-07-18
HopChain: Multi-Hop Data Synthesis for Generalizable Vision-Language Reasoning76.47Qwen3.5-397B-A17B (RLVR w/ Multi-Hop)2026-03-17
Qwen3.6✓ Link75.8Qwen3.6-27B2026-04-21
Qwen3.6✓ Link75.3Qwen3.6-35B-A3B2026-04-21
Qwen3.5: Towards Native Multimodal Agents✓ Link75.1Qwen3.5-35B-A3B2026-02-16
Qwen3.5: Towards Native Multimodal Agents✓ Link75.0Qwen3.5-27B2026-02-16
Inkling-Small✓ Link74.0Inkling-Small2026-07-27
Muse Glimmer 30B: An On-Device Agentic Model✓ Link74.0Muse Glimmer-30B (High Reasoning)2026-10-17
Qwen3.5-Omni Technical Report73.9Qwen3.5-Omni-Plus2026-04-21
Scalable Visual Pretraining for Language Intelligence73.87Qwen3.5-35B-A3B + VP2026-07-10
Gemma 4 Technical Report✓ Link73.8Gemma 4 26B-A4B2026-07-02
Inkling: Our open-weights model✓ Link73.5Inkling2026-07-15
SenseNova-U1: Unifying Multimodal Understanding and Generation with NEO-unify Architecture✓ Link72.83SenseNova-U1-A3B-MoT2026-05-12
Intern-S1-Pro: Scientific Multimodal Foundation Model at Trillion Scale72.8Intern-S1-Pro2026-03-26
Disaggregated Quantization: Specializing LLM Prefill and Decode✓ Link72.31Qwen3.8-27B NVFP4 prefiller + Q3_K_XL decoder (extra data)2026-09-22
Qwen3.5: Towards Native Multimodal Agents✓ Link70.1Qwen3.5-9B2026-02-16
Qwen3-VL Technical Report✓ Link69.3Qwen3-VL-235B-A22B-Thinking2025-11-26
Gemma 4 Technical Report✓ Link69.1Gemma 4 12B Unified2026-07-02
Announcing Bonsai 27B: The First 27B-Class Model to Run on a Phone✓ Link68.96Ternary Bonsai 27B2026-07-14
ERNIE 5.0 Technical Report68.63ERNIE 5.02026-02-04
EXAONE 4.5 Technical Report✓ Link68.6EXAONE 4.5 (Reasoning)2026-04-09
OmniEdu: Open Foundation Models for Learning and Teaching✓ Link67.98OmniEdu-27B2026-09-19
STEP3-VL-10B Technical Report✓ Link67.18Step3-VL-10B (PaCoRe)2026-01-14
SenseNova-U1.5: Towards Native Unified Visual Intelligence✓ Link66.47SenseNova-U1.5 8B-Think2026-09-10
GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learning✓ Link66.0GLM-4.6V2025-07-01
Command A+63.0Command A+2026-05-20
Masking Teacher and Reinforcing Student for Distilling Vision-Language Models61.4Qwen3-VL - Masters2025-12-23
Qwen3-Omni Technical Report✓ Link60.8Qwen3-Omni-Flash-Thinking2025-09-22
LongCat-Next: Lexicalizing Modalities as Discrete Tokens✓ Link60.3LongCat-Next (A3B, 68.5B)2026-03-29
The Llama 4 herd: The beginning of a new era of natively multimodal AI innovation✓ Link59.6Llama 4 Maverick2025-04-05
EvoCUA: Evolving Computer Use Agents via Learning from Scalable Synthetic Experience✓ Link59.16EvoCUA-32B2026-01-22
ERNIE 4.5 Technical Report✓ Link58.8ERNIE-4.5-VL-424B-A47B2025-06-30
V-Rubrics: Visual Faithfulness via Rubric-Based Reinforcement Learning✓ Link58.15V-Rubrics (GRPO w/ rubrics) (extra data)2026-08-26
Context-Aware RL for Agentic and Multimodal LLMs✓ Link57.5ContextRL (trained from Qwen3-VL-8B)2026-06-15
Near-Future Policy Optimization57.24AutoNPO2026-04-22
Penguin-VL: Exploring the Efficiency Limits of VLM with LLM-based Vision Encoders✓ Link55.9Penguin-VL 8B2026-03-06
Guidance Contrastive Token Credit Assignment for Discrete Policy Optimization✓ Link55.3Qwen3-VL-Instruct-8B + GCPO2026-05-28
Co-Evolving Policy Distillation55.12CoPD2026-04-29
Ovis2.5 Technical Report✓ Link54.4Ovis2.5-9B2025-08-15
DiffusionGemma Technical Report✓ Link54.3DiffusionGemma 26B A4B2026-07-31
Zone of Proximal Policy Optimization: Teacher in Prompts, Not Gradients53.2ZPPO (Qwen3.5-2B)2026-06-16
MiniMax-01: Scaling Foundation Models with Lightning Attention✓ Link52.7MiniMax-VL-012025-01-14
The Llama 4 herd: The beginning of a new era of natively multimodal AI innovation✓ Link52.2Llama 4 Scout2025-04-05
Qwen2.5-VL Technical Report✓ Link51.1Qwen2.5-VL-72B2025-02-19
Not All Prompts Are Equal: Exploration-Guided Prompt Scaffolding for Multimodal Reinforcement Post-Training50.29Qwen3-VL-4B + EPS (MMK12)2026-09-14
Cosmos 3: Omnimodal World Models for Physical AI✓ Link48.1Cosmos3-Super2026-06-01
Kimi-VL Technical Report✓ Link46.3Kimi-VL-A3B-Thinking-25062025-04-10
Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution✓ Link46.2Qwen2-VL-72B2024-09-18
Insight-V++: Towards Advanced Long-Chain Visual Reasoning with Multimodal Large Language Models45.6Insight-V++2026-03-18
EmbodiedMemory-Bench: Benchmarking Embodied Memory for Long-Horizon Embodied Tasks✓ Link45.1EMem-8B2026-09-23
Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution✓ Link43.5Qwen2-VL-7B2024-09-18
Learning Adaptive Reasoning Paths for Efficient Visual Reasoning✓ Link43.5AVR-8B2026-04-16
From Narrow to Panoramic Vision: Attention-Guided Cold-Start Reshapes Multimodal Reasoning✓ Link42.9AVAR-Thinker2026-03-04
VideoAuto-R1: Video Auto Reasoning via Thinking Once, Answering Twice✓ Link39.8VideoAuto-R1 (Qwen2.5-VL-7B)2026-01-08
Harnessing Textual Refusal Directions for Multimodal Safety✓ Link39.3MARS (Qwen3-VL 8B)2026-10-16
LoopVL: Recurrent Visual Intelligence✓ Link38.67LoopVL-1B2026-09-29
Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs✓ Link38.5Phi-4-Multimodal2025-02-24
Intern Lumina U2: A Multi-Codebook Diffusion Large Language Model for Omni-Visual Understanding and Image Generation✓ Link38.5Intern Lumina U2 (Ascend)2026-10-14
Qwen2.5-VL Technical Report✓ Link38.3Qwen2.5-VL-7B2025-02-19
Reinforcement Learning for Multimodal Diffusion Language Models via Bidimensional Trajectory and Thought Optimization38.3LLaDA-V + Bi-VRL (ours)2026-09-08
Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution✓ Link37.6Qwen2-VL-2B2024-09-18
Penguin-VL: Exploring the Efficiency Limits of VLM with LLM-based Vision Encoders✓ Link36.5Penguin-VL 2B2026-03-06
LoMo: Local Modality Substitution for Deeper Vision-Language Fusion✓ Link36.36LoMo (LLaVA-OneVision-1.5-8B)2026-05-28
VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understanding✓ Link33.6VideoLLaMA3 7B2025-01-22
Information-Regularized Attention for Visual-Centric Reasoning31.2InternVL2-IRA2026-07-01
VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understanding✓ Link28.6VideoLLaMA3 2B2025-01-22
PixelUMM: Encoder-Free Unified Image and Video Understanding and Generation✓ Link27.63PixelUMM 8B MoT2026-09-29
Dream-VL & Dream-VLA: Open Vision-Language and Vision-Language-Action Models with Diffusion Language Model Backbone✓ Link26.0Dream-VL2025-12-27
Check out how to contribute new results. Then edit this file on GitLab or GitHub.