CoCoT: Contrastive Chain-of-Thought Prompting for Large Multimodal Models with Multiple Image Inputs

Benchmark Model Rank Results
visual-reasoning-on-winogroundMMICL + CoCoT#1Text Score: 64.25Image Score: 52.5Group Score: 50.75
visual-reasoning-on-winogroundGPT-4V + CoCoT#2Text Score: 58.5Image Score: 49.5Group Score: 44.5
visual-reasoning-on-winogroundOpenFlamingo + CoCoT#3Text Score: 58.25Image Score: 55.25Group Score: 41.5
visual-reasoning-on-winogroundGPT-4V#4Text Score: 54.5Image Score: 42.5Group Score: 37.75
visual-reasoning-on-winogroundMMICL + CCoT#7Text Score: 51Image Score: 48Group Score: 47.5
visual-reasoning-on-winogroundOpenFlamingo + DDCoT#8Text Score: 47.5Image Score: 47.25Group Score: 39
visual-reasoning-on-winogroundMMICL + DDCoT#10Text Score: 46.75Image Score: 45Group Score: 36.75
visual-reasoning-on-winogroundGemini + DDCoT#17Text Score: 45Image Score: 25Group Score: 23.75
visual-reasoning-on-winogroundOpenFlamingo + CCoT#23Text Score: 42.5Image Score: 27.5Group Score: 20
visual-reasoning-on-winogroundGemini + CoCoT#25Text Score: 40Image Score: 32.5Group Score: 27.75
visual-reasoning-on-winogroundOpenFlamingo#27Text Score: 39Image Score: 41.25Group Score: 33.25
visual-reasoning-on-winogroundGemini#45Text Score: 30.75Image Score: 26Group Score: 25
visual-reasoning-on-winogroundGemini + CCoT#64Text Score: 22.5Image Score: 33Group Score: 20.75