T5Gemma 2: Seeing, Reading, and Understanding Longer

Benchmark Model Rank Results
arithmetic-reasoning-on-gsm8kT5Gemma 2 4B-4B (post-trained)–Accuracy: 88.6
chart-question-answering-on-chartqaT5Gemma 2 4B-4B (post-trained)–1:1 Accuracy: 75.0
code-generation-on-humanevalT5Gemma 2 4B-4B (post-trained)–Pass@1: 76.8
code-generation-on-mbppT5Gemma 2 4B-4B (post-trained)–Accuracy: 66.8
common-sense-reasoning-on-arc-easyT5Gemma 2 4B-4B–Accuracy: 74.3
common-sense-reasoning-on-winograndeT5Gemma 2 4B-4B–Accuracy: 71.6
math-word-problem-solving-on-mathT5Gemma 2 4B-4B–Accuracy: 21.2
multi-task-language-understanding-on-mmlu-proT5Gemma 2 4B-4B (post-trained)–Accuracy: 44.4
multimodal-reasoning-on-mmmuT5Gemma 2 4B-4B (post-trained)–Accuracy: 47.8
multiple-choice-question-answering-mcqa-on-gpqa-diamondT5Gemma 2 4B-4B (post-trained)–Accuracy: 27.8
question-answering-on-boolqT5Gemma 2 4B-4B–Accuracy: 79.3
question-answering-on-natural-questionsT5Gemma 2 4B-4B–EM: 17.2
question-answering-on-piqaT5Gemma 2 4B-4B–Accuracy: 79.2
sentence-completion-on-hellaswagT5Gemma 2 4B-4B–Accuracy: 77.4
visual-question-answering-on-docvqa-testT5Gemma 2 4B-4B (post-trained)–ANLS: 0.752
visual-question-answering-vqa-on-realworldqaT5Gemma 2 4B-4B–Accuracy: 46.1