| arithmetic-reasoning-on-gsm8k | Gemini Pro (maj1@32) | #28 | Accuracy: 86.5 |
| chart-question-answering-on-chartqa | Gemini Ultra | #1 | 1:1 Accuracy: 80.8 |
| long-context-understanding-on-mmneedle | Gemini Pro 1.0 | #4 | 1 Image, 4*4 Stitching, Exact Accuracy: 24.78… |
| math-word-problem-solving-on-math | Gemini Ultra (4-shot) | #32 | Accuracy: 53.2 |
| math-word-problem-solving-on-math | Gemini Pro (4-shot) | #72 | Accuracy: 32.6 |
| temporal-casual-qa-on-next-qa | Gemini Ultra (zero-shot) | #4 | WUPS: 29.9 |
| temporal-casual-qa-on-next-qa | Gemini Pro (zero-shot) | #5 | WUPS: 28.0 |
| visual-question-answering-on-mm-vet | Gemini 1.0 Pro Vision (gemini-pro-vision) | #17 | GPT-4 score: 64.3±0.4 |
| visual-question-answering-on-mm-vet-v2 | Gemini Pro Vision | #7 | GPT-4 score: 57.2±0.2 |
| visual-question-answering-vqa-on-ai2d | Gemini Ultra | #1 | EM: 79.5 |
| visual-question-answering-vqa-on-infographicvqa | Gemini Ultra (pixel only) | #1 | ANLS: 80.3 |