Rethinking Projector Training in Multimodal LLMs

Benchmark Model Rank Results
visual-question-answering-vqa-on-blinkPORTAL (Qwen 2.5 7B + CLIP)#49Accuracy: 41.5
visual-question-answering-vqa-on-realworldqaPORTAL (LLaMA 3.1 8B + CLIP)#86Accuracy: 56.9
chart-question-answering-on-chartqaPORTAL (Vicuna 1.5 7B + SigLIP)#1121:1 Accuracy: 17.9
multimodal-reasoning-on-mathvistaPORTAL (Qwen 2.5 7B + CLIP)#113Accuracy: 31.9
multimodal-reasoning-on-mmmuPORTAL (Qwen 2.5 7B + CLIP)#113Accuracy: 44.8