X$^2$-VLM: All-In-One Pre-trained Model For Vision-Language Tasks

Benchmark Model Rank Results
cross-modal-retrieval-on-flickr30kX2-VLM (large)#1Image-to-text R@1: 98.8Image-to-text R@5: 100Image-to-text R@10: 100…
cross-modal-retrieval-on-flickr30kX2-VLM (base)#2Image-to-text R@1: 98.5Image-to-text R@5: 100Image-to-text R@10: 100…
cross-modal-retrieval-on-coco-2014X2-VLM (large)#2Text-to-image R@1: 67.7Text-to-image R@5: 87.5…
cross-modal-retrieval-on-coco-2014X2-VLM (base)#5Text-to-image R@1: 66.2Text-to-image R@5: 87.1…
visual-reasoning-on-nlvr2-devX2-VLM (large)#2Accuracy: 88.7
visual-reasoning-on-nlvr2-devX2-VLM (base)#4Accuracy: 86.2
visual-grounding-on-refcoco-plus-test-bX2-VLM (large)#3Accuracy (%): 81.8
visual-grounding-on-refcoco-plus-test-bX2-VLM (base)#5Accuracy (%): 78.4
visual-grounding-on-refcoco-plus-testaX2-VLM (large)#3Accuracy (%): 92.1
visual-grounding-on-refcoco-plus-testaX2-VLM (base)#5Accuracy (%): 90.3
visual-grounding-on-refcoco-plus-valX2-VLM (large)#3Accuracy (%): 87.6
visual-grounding-on-refcoco-plus-valX2-VLM (base)#5Accuracy (%): 85.2
visual-reasoning-on-nlvr2-testX2-VLM (large)#3Accuracy: 89.4
visual-reasoning-on-nlvr2-testX2-VLM (base)#6Accuracy: 87.0
visual-question-answering-on-vqa-v2-test-stdX2-VLM (large)#6overall: 81.8
visual-question-answering-on-vqa-v2-test-stdX2-VLM (base)#9overall: 80.2
visual-question-answering-on-vqa-v2-test-devX2-VLM (large)#8Accuracy: 81.9
visual-question-answering-on-vqa-v2-test-devX2-VLM (base)#11Accuracy: 80.4
visual-question-answering-on-msrvtt-qaX2-VLM (large)#11Accuracy: 0.455
visual-question-answering-on-msrvtt-qaX2-VLM (base)#12Accuracy: 0.45
visual-question-answering-on-msvd-qaX2-VLM (large)#13Accuracy: 0.546
visual-question-answering-on-msvd-qaX2-VLM (base)#14Accuracy: 0.528
video-retrieval-on-msr-vtt-1kaX2-VLM (large)#14text-to-video R@1: 49.6text-to-video R@5: 76.7…
video-retrieval-on-msr-vtt-1kaX2-VLM (base)#25text-to-video R@1: 47.6text-to-video R@5: 74.1…