An Examination of the Compositionality of Large Generative Vision-Language Models

Open paper
Benchmark Model Rank Results
visual-reasoning-on-winogroundLLaVA-7B (GPTScore)#57Text Score: 25.50Image Score: 17.00Group Score: 10.50
visual-reasoning-on-winogroundMiniGPT-4-7B (GPTScore)#60Text Score: 24.50Image Score: 21.75Group Score: 11.50
visual-reasoning-on-winogroundMiniGPT-4-7B (VisualGPTScore)#62Text Score: 23.25Image Score: 18.00Group Score: 9.50
visual-reasoning-on-winogroundMiniGPT-4-7B (BERTScore)#75Text Score: 14.00Image Score: 8.00Group Score: 2.75
visual-reasoning-on-winogroundLLaVA-7B (BERTScore)#76Text Score: 13.50Image Score: 5.25Group Score: 2.25

Rank counts only results with a code link.