The effectiveness of MAE pre-pretraining for billion-scale pretraining

Open paper
Benchmark Model Rank Results
few-shot-image-classification-on-imagenet-10-shotMAWS (ViT-6.5B)#1Top 1 Accuracy: 84.6
few-shot-image-classification-on-imagenet-10-shotMAWS (ViT-2B)#3Top 1 Accuracy: 83.7
few-shot-image-classification-on-imagenet-10-shotMAWS (ViT-H)#4Top 1 Accuracy: 82.5
few-shot-image-classification-on-imagenet-1-shotMAWS (ViT-6.5B)#2Top 1 Accuracy: 63.6
few-shot-image-classification-on-imagenet-1-shotMAWS (ViT-2B)#7Top 1 Accuracy: 62.1
few-shot-image-classification-on-imagenet-1-shotMAWS (ViT-H)#8Top 1 Accuracy: 57.1
few-shot-image-classification-on-imagenet-5-shotMAWS (ViT-6.5B)#2Top 1 Accuracy: 82.6
few-shot-image-classification-on-imagenet-5-shotMAWS (ViT-2B)#3Top 1 Accuracy: 81.5
few-shot-image-classification-on-imagenet-5-shotMAWS (ViT-H)#4Top 1 Accuracy: 79.8
image-classification-on-inaturalist-2018MAWS (ViT-2B)#2Top-1 Accuracy: 91.3%
zero-shot-transfer-image-classification-on-food-101MAWS (ViT-2B)#4Top 1 Accuracy: 96.2
image-classification-on-imagenet-v2MAWS (ViT-6.5B)#5Top 1 Accuracy: 84.0
image-classification-on-imagenet-v2MAWS (ViT-2B)#7Top 1 Accuracy: 83.0
image-classification-on-imagenet-realMAWS (ViT-6.5B)#6Accuracy: 91.1%
image-classification-on-imagenet-realMAWS (ViT-2B)#11Accuracy: 90.9%
image-classification-on-imagenet-realMAWS (ViT-H)#14Accuracy: 90.8%
zero-shot-transfer-image-classification-on-imagenetMAWS (ViT-2B)#8Accuracy (Private): 82.1
zero-shot-transfer-image-classification-on-imagenetMAWS (ViT-H)#10Accuracy (Private): 81.1
image-classification-on-imagenetMAWS (ViT-6.5B)#10Top 1 Accuracy: 90.1%Number of params: 6500M
image-classification-on-imagenetMAWS (ViT-2B)#15Top 1 Accuracy: 89.8%Number of params: 2000M
image-classification-on-imagenetMAWS (ViT-H)#21Top 1 Accuracy: 89.5%Number of params: 650M
image-classification-on-imagenetMAWS (ViT-L)#35Top 1 Accuracy: 88.8%
image-classification-on-imagenetMAWS (ViT-B)#131Top 1 Accuracy: 86.8%
image-classification-on-objectnetMAWS (ViT-6.5B)#11Top-1 Accuracy: 77.9
image-classification-on-objectnetMAWS (ViT-2B)#13Top-1 Accuracy: 75.8
image-classification-on-objectnetMAWS (ViT-H)#14Top-1 Accuracy: 72.6
action-recognition-in-videos-on-something-something-v2MAWS (ViT-L)#14Top-1 Accuracy: 74.4

Rank counts only results with a code link.