Incorporating Structured Representations into Pretrained Vision & Language Models Using Scene Graphs

Benchmark Model Rank Results
visual-reasoning-on-winogroundBLIP–Text Score: 39.0Image Score: 19.2Group Score: 15.0
visual-reasoning-on-winogroundBLIP (+Graph Text)–Text Score: 40.3Image Score: 20.5Group Score: 16.5
visual-reasoning-on-winogroundBLIP (+Graph Text, +Graph Neg)–Text Score: 40.5Image Score: 25.5Group Score: 19.0
visual-reasoning-on-winogroundBLIP (SGVL)–Text Score: 42.8Image Score: 27.3Group Score: 21.5
visual-reasoning-on-winogroundBLIP2–Text Score: 42.0Image Score: 23.8Group Score: 19.0
visual-reasoning-on-winogroundBLIP2 (SGVL)–Text Score: 42.8Image Score: 28.5Group Score: 23.3
visual-reasoning-on-winogroundCLIP (SGVL)–Text Score: 32.0Image Score: 14.0Group Score: 9.8
visual-reasoning-on-winogroundLLaVA–Text Score: 24.8Image Score: 25.0Group Score: 13.0
visual-reasoning-on-winogroundMiniGPT-4–Text Score: 23.3Image Score: 18.0Group Score: 9.5
visual-reasoning-on-winogroundNegBLIP–Text Score: 42.5Image Score: 24.0Group Score: 18.5
visual-reasoning-on-winogroundNegBLIP2–Text Score: 41.5Image Score: 26.0Group Score: 20.5
visual-reasoning-on-winogroundNegCLIP–Text Score: 29.5Image Score: 10.5Group Score: 8.0