Flamingo: a Visual Language Model for Few-Shot Learning

Benchmark Model Rank Results
action-recognition-on-rareact馃Ι Flamingo#1mWAP: 60.8
generative-visual-question-answering-on-pmcOpen-Flamingo#3BLEU-1: 4.1
meme-classification-on-hateful-memesFlamingo (fine-tuned)#5ROC-AUC: 0.866
meme-classification-on-hateful-memesFlamingo (few-shot:32)#15ROC-AUC: 0.700
temporal-casual-qa-on-next-qaFlamingo(32-shot)#3WUPS: 33.5
temporal-casual-qa-on-next-qaFlamingo(0-shot)#6WUPS: 26.7
video-question-answering-on-situatedFlamingo-9B (4-shot)#13Average Accuracy: 42.8
video-question-answering-on-situatedFlamingo-80B (4-shot)#14Average Accuracy: 42.4
video-question-answering-on-situatedFlamingo-9B (0-shot)#15Average Accuracy: 41.8
video-question-answering-on-situatedFlamingo-80B (0-shot)#16Average Accuracy: 39.7
visual-question-answering-on-msrvtt-qa-1Flamingo#3Accuracy: 0.474
visual-question-answering-on-msrvtt-qa-1Flamingo (32-shot)#25Accuracy: 0.310
visual-question-answering-on-msrvtt-qa-1Flamingo (0-shot)#27Accuracy: 0.174
visual-question-answering-on-ok-vqaFlamingo80B#15Accuracy: 50.6
visual-question-answering-on-ok-vqaFlamingo9B#20Accuracy: 44.7
visual-question-answering-on-ok-vqaFlamingo3B#23Accuracy: 41.2
visual-question-answering-on-vqa-v2-test-devFlamingo 80B#46Accuracy: 56.3
visual-question-answering-on-vqa-v2-test-devFlamingo 9B#49Accuracy: 51.8
visual-question-answering-on-vqa-v2-test-devFlamingo 3B#51Accuracy: 49.2
visual-question-answering-vqa-on-pmc-vqaOpen-Flamingo#2Accuracy: 26.4
zero-shot-cross-modal-retrieval-on-coco-2014Flamingo#10Image-to-text R@1: 65.9Image-to-text R@5: 87.3
zero-shot-cross-modal-retrieval-on-flickr30kFlamingo#12Image-to-text R@1: 89.3Image-to-text R@5: 98.8
zero-shot-video-question-answer-on-starFlamingo-9B#3Accuracy: 41.8