X$^2$-VLM: All-In-One Pre-trained Model For Vision-Language Tasks

Benchmark Model Rank Results
cross-modal-retrieval-on-coco-2014X2-VLM (large)#2Text-to-image R@1: 67.7Text-to-image R@5: 87.5
cross-modal-retrieval-on-coco-2014X2-VLM (base)#5Text-to-image R@1: 66.2Text-to-image R@5: 87.1
cross-modal-retrieval-on-flickr30kX2-VLM (large)#1Image-to-text R@1: 98.8Image-to-text R@5: 100Image-to-text R@10: 100
cross-modal-retrieval-on-flickr30kX2-VLM (base)#2Image-to-text R@1: 98.5Image-to-text R@5: 100Image-to-text R@10: 100
video-retrieval-on-msr-vtt-1kaX2-VLM (large)#13text-to-video R@1: 49.6text-to-video R@5: 76.7
video-retrieval-on-msr-vtt-1kaX2-VLM (base)#23text-to-video R@1: 47.6text-to-video R@5: 74.1
visual-grounding-on-refcoco-test-bX2-VLM (large)#3Accuracy (%): 81.8
visual-grounding-on-refcoco-test-bX2-VLM (base)#5Accuracy (%): 78.4
visual-grounding-on-refcoco-testaX2-VLM (large)#3Accuracy (%): 92.1
visual-grounding-on-refcoco-testaX2-VLM (base)#5Accuracy (%): 90.3
visual-grounding-on-refcoco-valX2-VLM (large)#3Accuracy (%): 87.6
visual-grounding-on-refcoco-valX2-VLM (base)#5Accuracy (%): 85.2
visual-question-answering-on-msrvtt-qa-1X2-VLM (large)#10Accuracy: 0.455
visual-question-answering-on-msrvtt-qa-1X2-VLM (base)#11Accuracy: 0.45
visual-question-answering-on-msvd-qa-1X2-VLM (large)#13Accuracy: 0.546
visual-question-answering-on-msvd-qa-1X2-VLM (base)#14Accuracy: 0.528
visual-question-answering-on-vqa-v2-test-devX2-VLM (large)#7Accuracy: 81.9
visual-question-answering-on-vqa-v2-test-devX2-VLM (base)#9Accuracy: 80.4
visual-question-answering-on-vqa-v2-test-stdX2-VLM (large)#4overall: 81.8
visual-question-answering-on-vqa-v2-test-stdX2-VLM (base)#7overall: 80.2
visual-reasoning-on-nlvr2-devX2-VLM (large)#2Accuracy: 88.7
visual-reasoning-on-nlvr2-devX2-VLM (base)#4Accuracy: 86.2
visual-reasoning-on-nlvr2-testX2-VLM (large)#2Accuracy: 89.4
visual-reasoning-on-nlvr2-testX2-VLM (base)#5Accuracy: 87.0