Bongard-OpenWorld: Few-Shot Reasoning for Free-form Visual Concepts in the Real World

Benchmark Model Rank Results
visual-reasoning-on-bongard-openworldHuman#12-Class Accuracy: 91.0
visual-reasoning-on-bongard-openworldSNAIL#22-Class Accuracy: 64.0
visual-reasoning-on-bongard-openworldInstructBLIP + GPT-4#32-Class Accuracy: 63.8
visual-reasoning-on-bongard-openworldBLIP-2 + ChatGPT (Fine-tuned)#42-Class Accuracy: 63.3
visual-reasoning-on-bongard-openworldInstructBLIP + ChatGPT + Neuro-Symbolic#52-Class Accuracy: 55.5
visual-reasoning-on-bongard-openworldChatCaptioner + ChatGPT#62-Class Accuracy: 49.3
visual-reasoning-on-bongard-openworldOtter#72-Class Accuracy: 49.3