ERNIE 4.5 Technical Report

Benchmark Model Rank Results
question-answering-on-simpleqaERNIE-4.5-300B-A47B#1Accuracy: 45.9
visual-question-answering-vqa-on-mmbench-cnERNIE-4.5-VL-424B-A47B#2Accuracy: 90.9
instruction-following-on-multi-ifERNIE-4.5-300B-A47B#4Accuracy: 76.6
multi-task-language-understanding-on-livebench-2024-11-25ERNIE-4.5-300B-A47B#5Score: 68.9
multi-task-language-understanding-on-c-evalERNIE-4.5-300B-A47B#6Accuracy: 90.6
arithmetic-reasoning-on-gsm8kERNIE-4.5-300B-A47B#8Accuracy: 96.6
logic-puzzle-solving-on-zebralogicERNIE-4.5-300B-A47B#10Accuracy: 58.1
object-counting-on-countbenchERNIE-4.5-VL-424B-A47B#16Accuracy: 89.0
mathematical-reasoning-on-math-500ERNIE-4.5-300B-A47B#17Accuracy: 96.4
optical-character-recognition-ocr-on-ocrbenchERNIE-4.5-VL-424B-A47B#19Score: 872
code-generation-on-humanevalERNIE-4.5-300B-A47B#24Pass@1: 92.1
chart-question-answering-on-chartqaERNIE-4.5-VL-424B-A47B#271:1 Accuracy: 86.8
multimodal-reasoning-on-mmmu-proERNIE-4.5-VL-424B-A47B#32Accuracy: 58.8
visual-question-answering-vqa-on-realworldqaERNIE-4.5-VL-424B-A47B#34Accuracy: 73.2
multimodal-reasoning-on-mathvistaERNIE-4.5-VL-424B-A47B#35Accuracy: 78.9
multimodal-reasoning-on-mmmuERNIE-4.5-VL-424B-A47B#36Accuracy: 70.0
multi-task-language-understanding-on-mmlu-proERNIE-4.5-300B-A47B#46Accuracy: 78.4
code-generation-on-livecodebench-v6ERNIE-4.5-300B-A47B#57Pass@1: 38.8
mathematical-reasoning-on-aime24ERNIE-4.5-300B-A47B#75Acc: 54.8
mathematical-reasoning-on-aime-2025ERNIE-4.5-300B-A47B#107Accuracy: 35.1