Learning Transferable Visual Models From Natural Language Supervision

Benchmark Model Rank Results
object-categorization-on-gritCLIP#1Categorization (ablation): 48.1
action-recognition-on-rareactCLIP#2mWAP: 40.7
few-shot-image-classification-on-imagenet-0-shotCLIP (ViT B/32)#2Accuracy: 63.2%
few-shot-image-classification-on-imagenet-0-shotCLIP (ResNet50)#3Accuracy: 59.6%
long-tail-learning-on-coco-mltCLIP(ViT-B/16)#2Average mAP: 60.17
long-tail-learning-on-coco-mltCLIP(ResNet-50)#4Average mAP: 56.19
long-tail-learning-on-voc-mltCLIP(ViT-B/16)#2Average mAP: 85.77
long-tail-learning-on-voc-mltCLIP(ResNet-50)#4Average mAP: 84.30
meme-classification-on-multioffCLIP#3Accuracy: 62.4F1: 48.1
semi-supervised-image-classification-on-imagenet-0-2-labeled-dataCLIP (ResNet-50)#3ImageNet Top-1 Accuracy: 40%
text-based-person-retrieval-with-noisy-correspondence-on-cuhk-pedesCLIP-C#4Rank-1: 66.41Rank 10: 90.89Rank-5: 85.15mAP: 59.36mINP: 43.02
text-based-person-retrieval-with-noisy-correspondence-on-icfg-pedesCLIP-C#4Rank 1: 55.25Rank-10: 81.32Rank-5: 74.76mAP: 31.09mINP: 4.94
text-based-person-retrieval-with-noisy-correspondence-on-rstpreidCLIP-C#4Rank 1: 54.45Rank 10: 86.70Rank 5: 77.80mAP: 42.58mINP: 21.38
hateful-meme-classification-on-harm-pCLIP#5Accuracy: 80.6F1: 80.3
image-classification-on-omnibenchmarkCLIP-RN50#5Average Top-1 Accuracy: 42.1
image-to-text-retrieval-on-cocoCLIP (zero-shot)#5Recall@1: 58.4Recall@5: 81.5Recall@10: 88.1
object-recognition-on-shape-biasCLIP (ViT-B)#6shape bias: 79.9
zero-shot-transfer-image-classification-on-sunCLIP#6Accuracy: 58.5
hateful-meme-classification-on-pridemmCLIP (fine-tuned)#7Accuracy: 72.4F1: 72.3
open-vocabulary-attribute-detection-on-ovad-boxCLIP VIT-B16#7mean average precision: 16.6
zero-shot-transfer-image-classification-on-objectnetCLIP#8Accuracy (Private): 72.3
zero-shot-transfer-image-classification-on-imagenet-v2CLIP#9Accuracy (Private): 70.1
zero-shot-transfer-image-classification-on-imagenet-aCLIP#12Accuracy (Private): 77.2
zero-shot-transfer-image-classification-on-imagenet-rCLIP#12Accuracy: 88.9
zero-shot-transfer-image-classification-on-imagenetCLIP(ViT-L/14-336px)#13Accuracy (Private): 76.2
zero-shot-transfer-image-classification-on-imagenetCLIP (ResNet50)#17Accuracy (Private): 59.6
zero-shot-transfer-image-classification-on-imagenetCLIP#18Accuracy (Public): 31.3
image-classification-on-objectnetCLIP#15Top-1 Accuracy: 72.3
prompt-engineering-on-ucf101CLIP#15Harmonic mean: 73.85
zero-shot-cross-modal-retrieval-on-flickr30kCLIP#17Image-to-text R@1: 88.0Image-to-text R@5: 98.7…
meme-classification-on-hateful-memesCLIP (zero-shot)#18ROC-AUC: 0.661
prompt-engineering-on-imagenet-v2CLIP#18Top-1 accuracy %: 60.83
prompt-engineering-on-imagenet-aCLIP#20Top-1 accuracy %: 47.77
prompt-engineering-on-imagenet-rCLIP#21Top-1 accuracy %: 73.96
prompt-engineering-on-imagenet-sCLIP#21Top-1 accuracy %: 46.15
zero-shot-cross-modal-retrieval-on-coco-2014CLIP#24Image-to-text R@1: 58.4Image-to-text R@5: 81.5…
prompt-engineering-on-caltech-101CLIP#28Harmonic mean: 95.40
prompt-engineering-on-eurosatCLIP#28Harmonic mean: 60.03
prompt-engineering-on-fgvc-aircraftCLIP#28Harmonic mean: 31.09
prompt-engineering-on-oxford-iiit-pet-datasetCLIP#28Harmonic mean: 94.12
prompt-engineering-on-stanford-carsCLIP#28Harmonic mean: 68.65
prompt-engineering-on-sun397CLIP#28Harmonic mean: 72.23
prompt-engineering-on-dtdCLIP#29Harmonic mean: 56.37
prompt-engineering-on-imagenetCLIP#29Harmonic mean: 70.22
prompt-engineering-on-oxford-102-flowerCLIP#29Harmonic mean: 74.83