PaLI: A Jointly-Scaled Multilingual Language-Image Model

Benchmark Model Rank Results
image-captioning-on-nocaps-in-domainPaLI#1CIDEr: 149.1
image-captioning-on-nocaps-in-domainPaLI#4CIDEr: 121.09B1: 88.02B2: 75.21B3: 59.38B4: 41.16
image-captioning-on-nocaps-near-domainPaLI#2CIDEr: 124.35B1: 88.57B2: 75.56B3: 58.99B4: 39.98
image-captioning-on-nocaps-near-domainPaLI#8SPICE: 15.75
image-captioning-on-nocaps-out-of-domainPaLI#1CIDEr: 126.67B1: 86.28B2: 71.19B3: 52.63B4: 32.0
image-classification-on-imagenet-v2ViT-e#2Top 1 Accuracy: 84.3
image-classification-on-objectnetViT-e#11Top-1 Accuracy: 72.0
visual-question-answering-on-ok-vqaPaLI 17B#3Accuracy: 64.5
visual-question-answering-on-textvqa-test-1PaLI#1overall: 73.1
visual-question-answering-on-vizwiz-2020-vqaPaLI#1overall: 73.3
visual-question-answering-on-vqa-v2-test-devPaLI#1Accuracy: 84.3
zero-shot-transfer-image-classification-on-1LiT ViT-e#4Accuracy (Private): 85.4
zero-shot-transfer-image-classification-on-1PaLI#15Accuracy (Private): 72.11
zero-shot-transfer-image-classification-on-3LiT ViT-e#3Accuracy (Private): 80.6
zero-shot-transfer-image-classification-on-3PaLI#11Accuracy (Private): 64.46
zero-shot-transfer-image-classification-on-4LiT ViT-e#2Accuracy: 96.1
zero-shot-transfer-image-classification-on-4PaLI#10Accuracy: 81.97
zero-shot-transfer-image-classification-on-5LiT ViT-e#3Accuracy (Private): 88.0
zero-shot-transfer-image-classification-on-5PaLI#11Accuracy (Private): 44.7
zero-shot-transfer-image-classification-on-6LiT ViT-e#2Accuracy (Private): 84.9
zero-shot-transfer-image-classification-on-6PaLI#9Accuracy (Private): 42.62Top 5 Accuracy: 58.35