COSMOS: Cross-Modality Self-Distillation for Vision Language Pre-training

Benchmark Model Rank Results
zero-shot-segmentation-on-ade20k-training-freeCOSMOS ViT-B/16#1mIoU: 17.7
unsupervised-semantic-segmentation-with-language-image-pre-training-on-cityscapes-valCOSMOS ViT-B/16#4mIoU: 34.7
zero-shot-cross-modal-retrieval-on-flickr30kCOSMOS ViT-B/16#5Image-to-text R@1: 92.9Image-to-text R@5: 99.4…
zero-shot-cross-modal-retrieval-on-flickr30kCOSMOS ViT-B/32#13Image-to-text R@1: 89.9Image-to-text R@5: 98.8…
unsupervised-semantic-segmentation-with-language-image-pre-training-on-ade20kCOSMOS ViT-B/16#6Mean IoU (val): 17.7
unsupervised-semantic-segmentation-with-language-image-pre-training-on-pascal-context-59COSMOS ViT-B/16#8mIoU: 33.7
unsupervised-semantic-segmentation-with-language-image-pre-training-on-coco-objectCOSMOS ViT-B/16#9mIoU: 31.3
unsupervised-semantic-segmentation-with-language-image-pre-training-on-coco-stuff-171COSMOS ViT-B/16#9mIoU: 23.2
unsupervised-semantic-segmentation-with-language-image-pre-training-on-pascalvoc-20COSMOS ViT-B/16#10mIoU: 77.7
zero-shot-cross-modal-retrieval-on-coco-2014COSMOS ViT-B/16#15Image-to-text R@1: 68.0Image-to-text R@5: 87.8…
zero-shot-cross-modal-retrieval-on-coco-2014COSMOS ViT-B/32#19Image-to-text R@1: 64.3Image-to-text R@5: 86.5…