| multi-task-language-understanding-on-mgsm | Gemini 1.5 Pro (8-shot) | #1 | Average (%): 87.5 |
| zero-shot-video-question-answer-on-video-mme | Gemini 1.5 Pro | #1 | Accuracy (%): 81.3 |
| zero-shot-video-question-answer-on-video-mme | Gemini 1.5 Flash | #3 | Accuracy (%): 75.0 |
| zero-shot-video-question-answer-on-longvideobench | Gemini 1.5 Pro | #2 | Accuracy (% ): 66.7 |
| zero-shot-video-question-answer-on-vnbench | Gemini | #2 | Accuracy: 66.7 |
| fs-mevqa-on-sme | Gemini-1.5 Pro | #3 | BLEU-4: 41.87METEOR: 34.61ROUGE-L: 55.90CIDEr: 276.14… |
| long-context-understanding-on-mmneedle | Gemini Pro 1.5 | #3 | 1 Image, 4*4 Stitching, Exact Accuracy: 39.85… |
| visual-question-answering-on-mm-vet-v2 | Gemini 1.5 Pro | #3 | GPT-4 score: 66.9±0.2 |
| video-question-answering-on-ovbench | Gemini-1.5-Flash | #4 | AVG: 50.7 |
| visual-question-answering-on-mm-vet | Gemini 1.5 Pro (gemini-1.5-pro-002) | #4 | GPT-4 score: 76.9±0.1 |
| visual-question-answering-on-mm-vet | Gemini 1.5 Pro (gemini-1.5-pro) | #22 | GPT-4 score: 65.8±0.1 |
| temporal-relation-extraction-on-vinoground | Gemini-1.5-Pro (CoT) | #5 | Text Score: 37Video Score: 27.6Group Score: 12.4 |
| temporal-relation-extraction-on-vinoground | Gemini-1.5-Pro | #7 | Text Score: 35.8Video Score: 22.6Group Score: 10.2 |
| zero-shot-video-question-answer-on-egoschema-fullset | Gemini 1.5 Pro | #5 | Accuracy: 72.2 |
| multi-task-language-understanding-on-mmlu | Gemini 1.5 Pro (5-shot) | #6 | Average (%): 85.9 |
| question-answering-on-newsqa | Google/Gemini 2.5 Pro | #7 | EM: 68.75F1: 79.91 |
| sentence-completion-on-hellaswag | Gemini 1.5 Pro (10-shot) | #10 | Accuracy: 93.3 |
| video-question-answering-on-tvbench | Gemini 1.5 Pro | #11 | Average Accuracy: 47.6 |
| visual-question-answering-vqa-on-textvqa-val | Gemini 1.5 Pro | #14 | Accuracy: 78.7 |
| zero-shot-video-question-answer-on-video-mme-w-o-subs | Gemini 1.5 Pro | #19 | Accuracy (%): 71.9 |
| zero-shot-video-question-answer-on-video-mme-w-o-subs | Gemini 1.5 Flash | #37 | Accuracy (%): 66.3 |
| visual-question-answering-on-docvqa-test | Gemini 1.5 Pro | #24 | ANLS: 0.931 |
| visual-question-answering-vqa-on-blink | Gemini 1.5 Pro | #25 | Accuracy: 61.4 |
| chart-question-answering-on-chartqa | Gemini 1.5 Pro | #27 | 1:1 Accuracy: 87.2 |
| math-word-problem-solving-on-math | Gemini 1.5 Pro (4-shot) | #39 | Accuracy: 67.7 |
| visual-question-answering-vqa-on-realworldqa | Gemini 1.5 Pro | #51 | Accuracy: 70.4 |
| multimodal-reasoning-on-mmmu | Gemini 1.5 Pro | #56 | Accuracy: 62.2 |
| arithmetic-reasoning-on-gsm8k | Gemini 1.5 Pro (11-shot) | #68 | Accuracy: 90.8 |
| multimodal-reasoning-on-mathvista | Gemini 1.5 Pro | #88 | Accuracy: 63.9 |
| code-generation-on-humaneval | Gemini 1.5 Pro | – | Pass@1: 84.1 |