Learning Transferable Visual Models From Natural Language Supervision

Benchmark Model Rank Results
action-recognition-on-rareactCLIP#2mWAP: 40.7
few-shot-image-classification-on-imagenet-0CLIP (ViT B/32)#2Accuracy: 63.2%
few-shot-image-classification-on-imagenet-0CLIP (ResNet50)#3Accuracy: 59.6%
hateful-meme-classification-on-harm-pCLIP#5Accuracy: 80.6F1: 80.3
hateful-meme-classification-on-pridemmCLIP (fine-tuned)#7Accuracy: 72.4F1: 72.3
image-classification-on-objectnetCLIP#9Top-1 Accuracy: 72.3
image-classification-on-omnibenchmarkCLIP-RN50#5Average Top-1 Accuracy: 42.1
image-to-text-retrieval-on-cocoCLIP (zero-shot)#5Recall@1: 58.4Recall@5: 81.5Recall@10: 88.1
long-tail-learning-on-coco-mltCLIP(ViT-B/16)#2Average mAP: 60.17
long-tail-learning-on-coco-mltCLIP(ResNet-50)#4Average mAP: 56.19
long-tail-learning-on-voc-mltCLIP(ViT-B/16)#2Average mAP: 85.77
long-tail-learning-on-voc-mltCLIP(ResNet-50)#4Average mAP: 84.30
meme-classification-on-hateful-memesCLIP (zero-shot)#16ROC-AUC: 0.661
meme-classification-on-multioffCLIP#3Accuracy: 62.4F1: 48.1
object-categorization-on-gritCLIP#1Categorization (ablation): 48.1
object-recognition-on-shape-biasCLIP (ViT-B)#6shape bias: 79.9
open-vocabulary-attribute-detection-on-ovad-1CLIP VIT-B16#7mean average precision: 16.6
prompt-engineering-on-caltech-101CLIP#14Harmonic mean: 95.40
prompt-engineering-on-dtdCLIP#14Harmonic mean: 56.37
prompt-engineering-on-eurosatCLIP#14Harmonic mean: 60.03
prompt-engineering-on-fgvc-aircraftCLIP#13Harmonic mean: 31.09
prompt-engineering-on-imagenetCLIP#15Harmonic mean: 70.22
prompt-engineering-on-imagenet-aCLIP#9Top-1 accuracy %: 47.77
prompt-engineering-on-imagenet-rCLIP#9Top-1 accuracy %: 73.96
prompt-engineering-on-imagenet-sCLIP#9Top-1 accuracy %: 46.15
prompt-engineering-on-imagenet-v2CLIP#7Top-1 accuracy %: 60.83
prompt-engineering-on-oxford-102-flowerCLIP#14Harmonic mean: 74.83
prompt-engineering-on-oxford-iiit-pet-datasetCLIP#14Harmonic mean: 94.12
prompt-engineering-on-stanford-cars-1CLIP#14Harmonic mean: 68.65
prompt-engineering-on-sun397CLIP#14Harmonic mean: 72.23
prompt-engineering-on-ucf101CLIP#14Harmonic mean: 73.85
semi-supervised-image-classification-on-16CLIP (ResNet-50)#3ImageNet Top-1 Accuracy: 40%
text-based-person-retrieval-with-noisyCLIP-C#4Rank-1: 66.41Rank 10: 90.89Rank-5: 85.15mAP: 59.36mINP: 43.02
text-based-person-retrieval-with-noisy-1CLIP-C#4Rank 1: 55.25Rank-10: 81.32Rank-5: 74.76mAP: 31.09mINP: 4.94
text-based-person-retrieval-with-noisy-2CLIP-C#4Rank 1: 54.45Rank 10: 86.70Rank 5: 77.80mAP: 42.58mINP: 21.38
zero-shot-cross-modal-retrieval-on-coco-2014CLIP#15Image-to-text R@1: 58.4Image-to-text R@5: 81.5
zero-shot-cross-modal-retrieval-on-flickr30kCLIP#15Image-to-text R@1: 88.0Image-to-text R@5: 98.7
zero-shot-transfer-image-classification-on-1CLIP(ViT-L/14-336px)#13Accuracy (Private): 76.2
zero-shot-transfer-image-classification-on-1CLIP (ResNet50)#17Accuracy (Private): 59.6
zero-shot-transfer-image-classification-on-1CLIP#18Accuracy (Public): 31.3
zero-shot-transfer-image-classification-on-2CLIP#2Accuracy: 58.5
zero-shot-transfer-image-classification-on-3CLIP#9Accuracy (Private): 70.1
zero-shot-transfer-image-classification-on-4CLIP#8Accuracy: 88.9
zero-shot-transfer-image-classification-on-5CLIP#8Accuracy (Private): 77.2
zero-shot-transfer-image-classification-on-6CLIP#8Accuracy (Private): 72.3