Scaling Vision Transformers to 22 Billion Parameters

Benchmark Model Rank Results
zero-shot-transfer-image-classification-on-imagenet-v2LiT-22B#1Accuracy (Private): 80.9
zero-shot-transfer-image-classification-on-objectnetLiT-22B#1Accuracy (Private): 87.6
zero-shot-transfer-image-classification-on-imagenetLiT-22B#3Accuracy (Private): 85.9
object-recognition-on-shape-biasViT-22B-384#4shape bias: 86.4
object-recognition-on-shape-biasViT-22B-560#5shape bias: 83.8
object-recognition-on-shape-biasViT-22B-224#7shape bias: 78.0
zero-shot-transfer-image-classification-on-imagenet-aLiT-22B#4Accuracy (Private): 90.1
zero-shot-transfer-image-classification-on-imagenet-rLiT-22B#4Accuracy: 96.0
image-classification-on-imagenetViT-L/16 (384res, distilled from ViT-22B)#19Top 1 Accuracy: 89.6%Number of params: 307M
image-classification-on-imagenetViT-B/16#42Top 1 Accuracy: 88.6%Number of params: 86M
action-classification-on-kinetics-400ViT-22B#25Acc@1: 88.0