OpenCodePapers

visual-question-answering-on-docvqa-test

Visual Question Answering (VQA)
Dataset Link
Results over time
Click legend items to toggle metrics. Hover points for model names.
Leaderboard
Click a sortable column header to sort.
PaperCodeANLSAccuracyModelNameReleaseDate
ORCA: Orchestrated Reasoning with Collaborative Agents for Document Visual Question Answering0.972ORCA (Qwen3VL-8B)2026-03-02
Qwen3-VL Technical Report✓ Link0.971Qwen3-VL-235B-A22B-Instruct2025-11-26
Visual Document Understanding and Question Answering: A Multi-Agent Collaboration Framework with Test-Time Scaling✓ Link0.966MACT Qwen2.5-VL Series 24B (Agent-Wise Adaptive TTS) (extra data)2025-08-05
Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution✓ Link0.965Qwen2-VL-72B2024-09-18
MiniMax-01: Scaling Foundation Models with Lightning Attention✓ Link0.964MiniMax-VL-012025-01-14
Qwen2.5-VL Technical Report✓ Link0.964Qwen2.5-VL-72B2025-02-19
ORCA: Orchestrated Reasoning with Collaborative Agents for Document Visual Question Answering0.964ORCA (Qwen2.5-VL-7B)2026-03-02
Penguin-VL: Exploring the Efficiency Limits of VLM with LLM-based Vision Encoders✓ Link0.961Penguin-VL 8B2026-03-06
ORCA: Orchestrated Reasoning with Collaborative Agents for Document Visual Question Answering0.96ORCA (Qwen3VL-4B)2026-03-02
Introducing Command A Vision: Multimodal AI built for business0.959Command A Vision2025-07-31
Nemotron 3 Nano Omni: Efficient and Open Multimodal Intelligence✓ Link0.956Nemotron 3 Nano Omni (Reasoning)2026-04-27
InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models✓ Link0.954InternVL3-78B2025-04-14
Introducing Claude 3.5 Sonnet0.952Claude 3.5 Sonnet2024-06-20
Qwen2.5-Omni Technical Report✓ Link0.952Qwen2.5-Omni-7B2025-03-26
LLaVA-OneVision-2: Towards Next-Generation Perceptual Intelligence✓ Link0.952LLaVA-OneVision-2-8B2026-05-25
Mage-VL: An Efficient Codec-Native Streaming Multimodal Foundation Model✓ Link0.9514Mage-VL-4B2026-07-27
VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understanding✓ Link0.949VideoLLaMA3 7B2025-01-22
MiniCPM-V 4.5: Cooking Efficient MLLMs via Architecture, Data, and Training Recipe✓ Link0.947MiniCPM-V 4.52025-09-16
NVIDIA Nemotron Nano V2 VL✓ Link0.947Nemotron Nano V2 VL (Reasoning off) (extra data)2025-11-06
Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution✓ Link0.945Qwen2-VL-7B2024-09-18
The Llama 4 herd: The beginning of a new era of natively multimodal AI innovation✓ Link0.944Llama 4 Scout2025-04-05
The Llama 4 herd: The beginning of a new era of natively multimodal AI innovation✓ Link0.944Llama 4 Maverick2025-04-05
DocVQA: A Dataset for VQA on Document Images✓ Link0.9436Human2020-07-01
Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Vision-Language Models✓ Link0.935Molmo 72B2024-09-25
Penguin-VL: Exploring the Efficiency Limits of VLM with LLM-based Vision Encoders✓ Link0.933Penguin-VL 2B2026-03-06
FastVLM: Efficient Vision Encoding for Vision Language Models✓ Link0.932FastVLM-7B2024-12-17
Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs✓ Link0.932Phi-4-Multimodal2025-02-24
Molmo2: Open Weights and Data for Vision-Language Models with Video Understanding and Grounding✓ Link0.932Molmo2-8B2026-01-15
Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context✓ Link0.931Gemini 1.5 Pro2024-03-08
Hello GPT-4o0.928GPT-4o2024-05-13
Qianfan-OCR: A Unified End-to-End Model for Document Intelligence✓ Link0.928Qianfan-OCR2026-03-11
Oryx MLLM: On-Demand Spatial-Temporal Understanding at Arbitrary Resolution✓ Link0.927Oryx-1.5 32B2024-09-19
The Llama 3 Herd of Models✓ Link0.926Llama 3-V (405B)2024-07-31
Aria: An Open Multimodal Native Mixture-of-Experts Model✓ Link0.926Aria2024-10-08
Beyond Correctness: Benchmarking and Aligning Response Behaviors in Hybrid-Thinking MLLMs0.9254Qwen3-VL-8B + PatternRL2026-08-17
ZAYA1-VL-8B Technical Report✓ Link0.925ZAYA1-VL-8B2026-05-08
VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understanding✓ Link0.919VideoLLaMA3 2B2025-01-22
From Pixels to Words -- Towards Native One-Vision Models at Scale✓ Link0.919NEO-ov 9B2026-05-27
Multimodal OCR: Parse Anything from Documents✓ Link0.9185dots.mocr2026-03-13
InfiniteVL: Synergizing Linear and Sparse Attention for Highly-Efficient, Unlimited-Input Vision-Language Models✓ Link0.917InfiniteVL-Base (extra data)2025-12-09
Multi-label Cluster Discrimination for Visual Representation Learning✓ Link0.916MLCD-Embodied-7B2024-07-24
LLaDA-UI: Bringing Block-wise Diffusion to Vision-Language GUI Agents✓ Link0.915LLaDA-UI (pre-training)2026-09-09
LLaVA-OneVision: Easy Visual Task Transfer✓ Link0.913LLaVA-OneVision-72B2024-08-06
Gemini: A Family of Highly Capable Multimodal Models✓ Link0.909Gemini Ultra2023-12-19
Omni-SMoLA: Boosting Generalist Multimodal Models with Soft Mixture of Low-rank Experts0.908SMoLA-PaLI-X Specialist2023-12-01
Pixtral 12B✓ Link0.907Pixtral 12B2024-10-09
Omni-SMoLA: Boosting Generalist Multimodal Models with Soft Mixture of Low-rank Experts0.906SMoLA-PaLI-X Generalist2023-12-01
Cosmos 3: Omnimodal World Models for Physical AI✓ Link0.904Cosmos3-Super2026-06-01
MoE-ViE: Mixture of Experts Vision Encoder for Efficient Image and Video Understanding✓ Link0.904MoE-ViE-H/14 + Llama 3.1 8B (42M alignment)2026-08-18
Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond✓ Link0.9024Qwen-VL-Plus2023-08-24
MODUS: Decoder-Only Any-to-Any Modeling of Diverse Modalities✓ Link0.9011Modus2026-07-28
Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution✓ Link0.901Qwen2-VL-2B2024-09-18
Llama 3.2: Revolutionizing edge AI and vision with open, customizable models✓ Link0.901Llama 3.2 90B Vision Instruct2024-09-25
From Pixels to Words -- Towards Native Vision-Language Primitives at Scale✓ Link0.899NEO-2.2B2025-10-16
ScreenAI: A Vision-Language Model for UI and Infographics Understanding✓ Link0.8988ScreenAI 5B (4.62 B params, w/OCR)2024-02-07
PaLI-3 Vision Language Models: Smaller, Faster, Stronger✓ Link0.886PaLI-3 (w/ OCR)2023-10-13
ERNIE-Layout: Layout Knowledge Enhanced Pre-training for Visually-rich Document Understanding✓ Link0.8841ERNIE-Layout large (ensemble)2022-10-12
GPT-4 Technical Report✓ Link0.884GPT-4 (vision, 0-shot pixel-only)2023-03-15
Layout and Task Aware Instruction Prompt for Zero-shot Document Image Question Answering✓ Link0.884GPT-42023-06-01
DocFormerv2: Local Features for Document Understanding✓ Link0.8784DocFormerv2-large2023-06-02
Unifying Vision, Text, and Layout for Universal Document Processing✓ Link0.878UDOP (aux)2022-12-05
Building and better understanding vision-language models: insights and future directions✓ Link0.877Idefics3-8B2024-08-22
PaLI-3 Vision Language Models: Smaller, Faster, Stronger✓ Link0.876PaLI-32023-10-13
Mini-Monkey: Alleviating the Semantic Sawtooth Effect for Lightweight MLLMs via Complementary Image Pyramid✓ Link0.874Mini-Monkey (extra data)2024-08-04
Going Full-TILT Boogie on Document Understanding with Text-Image-Layout Transformer✓ Link0.8705TILT-Large2021-02-18
PaLI-X: On Scaling up a Multilingual Vision and Language Model✓ Link0.868PaLI-X (Single-task FT w/ OCR)2023-05-29
LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understanding✓ Link0.8672LayoutLMv2LARGE2020-12-29
ERNIE-Layout: Layout Knowledge Enhanced Pre-training for Visually-rich Document Understanding✓ Link0.8486ERNIE-Layout large2022-10-12
PaliGemma: A versatile 3B VLM for transfer✓ Link0.848PaliGemma2024-07-10
MiniCPM-V: A GPT-4V Level MLLM on Your Phone✓ Link0.848MiniCPM-Llama3-V 2.52024-08-03
Unifying Vision, Text, and Layout for Universal Document Processing✓ Link0.847UDOP2022-12-05
Going Full-TILT Boogie on Document Understanding with Text-Image-Layout Transformer✓ Link0.8392TILT-Base2021-02-18
LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking✓ Link0.8337LayoutLMv3 Large2022-04-18
Layout and Task Aware Instruction Prompt for Zero-shot Document Image Question Answering✓ Link0.8336Claude + LATIN-Prompt2023-06-01
Layout and Task Aware Instruction Prompt for Zero-shot Document Image Question Answering✓ Link0.8255GPT-3.5 + LATIN-Prompt2023-06-01
mPLUG-DocOwl 1.5: Unified Structure Learning for OCR-free Document Understanding✓ Link0.822DocOwl 1.5 Chat (extra data)2024-03-19
KOSMOS-2.5: A Multimodal Literate Model✓ Link0.811Kosmos-2.5-chat2023-09-20
PaLI-X: On Scaling up a Multilingual Vision and Language Model✓ Link0.809PaLI-X (Multi-task FT)2023-05-29
mPLUG-DocOwl2: High-resolution Compressing for OCR-free Multi-page Document Understanding✓ Link0.807DocOwl22024-09-05
DUBLIN -- Document Understanding By Language-Image Network0.803DUBLIN (variable resolution)2023-05-23
PaLI-X: On Scaling up a Multilingual Vision and Language Model✓ Link0.80PaLI-X (Single-task FT)2023-05-29
SmolVLM: Redefining small and efficient multimodal models✓ Link0.800SmolVLM 2.2B2025-04-07
LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Models✓ Link0.789LLaVA-NeXT-Interleave 14B2024-07-10
DUBLIN -- Document Understanding By Language-Image Network0.782DUBLIN2023-05-23
LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understanding✓ Link0.7808LayoutLMv2BASE2020-12-29
Falcon2-11B Technical Report0.769Falcon2-11B VLM2024-07-20
Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understanding✓ Link0.766Pix2Struct-large2022-10-07
Emu3: Next-Token Prediction is All You Need✓ Link0.763Emu32024-09-27
On the Design Fundamentals of Pixel Text Representation Learning✓ Link0.7541Pixel Linguist II + Qwen2.5-7B-Instruct (30% visual tokens)2026-09-01
T5Gemma 2: Seeing, Reading, and Understanding Longer0.752T5Gemma 2 4B-4B (post-trained)2025-12-16
LayoutLLM: Layout Instruction Tuning with Large Language Models for Document Understanding✓ Link0.7427LayoutLLM 7B (Vicuna-1.5)2024-04-08
MatCha: Enhancing Visual Language Pretraining with Math Reasoning and Chart Derendering✓ Link0.742MatCha2022-12-19
What matters when building vision-language models?✓ Link0.740Idefics2 (w/ image split)2024-05-03
XDoc: Unified Pre-training for Cross-Format Document Understanding✓ Link0.727XDoc 1M2022-10-06
Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understanding✓ Link0.721Pix2Struct-base2022-10-07
VISion On Request: Enhanced VLLM efficiency with sparse, dynamically selected, vision-language interactions0.689VISOR2026-03-24
OCR-free Document Understanding Transformer✓ Link0.675Donut2021-11-30
DocVQA: A Dataset for VQA on Document Images✓ Link0.66555.77BERT_LARGE_SQUAD_DOCVQA_FINETUNED_Baseline2020-07-01
Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond✓ Link0.651Qwen-VL2023-08-24
End-to-end Document Recognition and Understanding with Dessurt✓ Link0.632Dessurt2022-03-30
Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond✓ Link0.626Qwen-VL-Chat2023-08-24
mPLUG-DocOwl: Modularized Multimodal Large Language Model for Document Understanding✓ Link0.622mPLUG-DocOwl (extra data)2023-07-04
Check out how to contribute new results. Then edit this file on GitLab or GitHub.