SimVLM: Simple Visual Language Model Pretraining with Weak Supervision

Benchmark Model Rank Results
image-captioning-on-coco-captionsSimVLM#16BLEU-4: 40.6CIDER: 143.3METEOR: 33.4SPICE: 25.4
image-captioning-on-nocaps-entireSingle Model#3CIDEr: 110.31B1: 83.78B2: 68.86B3: 51.06B4: 32.2
image-captioning-on-nocaps-in-domainSingle Model#5CIDEr: 108.98B1: 84.64B2: 70.0B3: 52.96B4: 34.66
image-captioning-on-nocaps-near-domainSingle Model#4CIDEr: 110.76B1: 84.36B2: 69.83B3: 52.42B4: 33.74
image-captioning-on-nocaps-out-of-domainSingle Model#4CIDEr: 109.49B1: 80.89B2: 64.21B3: 44.38B4: 24.47
image-captioning-on-nocaps-val-in-domainSimVLM#5CIDEr: 113.7Pre-train (#images): 1.8B
image-captioning-on-nocaps-val-near-domainSimVLM#5CIDEr: 110.9Pre-train (#images): 1.8B
image-captioning-on-nocaps-val-out-domainSimVLM#5CIDEr: 115.2Pretrain (#images): 1.8B
image-captioning-on-nocaps-val-overallSimVLM#5CIDEr: 112.2Pretrain (#images): 1.8B
visual-entailment-on-snli-ve-testSimVLM#4Accuracy: 86.32
visual-entailment-on-snli-ve-valSimVLM#4Accuracy: 86.21
visual-question-answering-on-vqa-v2-test-devSimVLM#11Accuracy: 80.03
visual-question-answering-on-vqa-v2-test-stdSimVLM#6overall: 80.34
visual-reasoning-on-nlvr2-devSimVLM#8Accuracy: 84.53
visual-reasoning-on-nlvr2-testSimVLM#7Accuracy: 85.15