The effectiveness of MAE pre-pretraining for billion-scale pretraining

Open paper
Benchmark Model Rank Results
action-recognition-in-videos-on-somethingMAWS (ViT-L)#14Top-1 Accuracy: 74.4
few-shot-image-classification-on-imagenet-1-1MAWS (ViT-6.5B)#2Top 1 Accuracy: 63.6
few-shot-image-classification-on-imagenet-1-1MAWS (ViT-2B)#7Top 1 Accuracy: 62.1
few-shot-image-classification-on-imagenet-1-1MAWS (ViT-H)#8Top 1 Accuracy: 57.1
few-shot-image-classification-on-imagenet-10MAWS (ViT-6.5B)#1Top 1 Accuracy: 84.6
few-shot-image-classification-on-imagenet-10MAWS (ViT-2B)#3Top 1 Accuracy: 83.7
few-shot-image-classification-on-imagenet-10MAWS (ViT-H)#4Top 1 Accuracy: 82.5
few-shot-image-classification-on-imagenet-5MAWS (ViT-6.5B)#2Top 1 Accuracy: 82.6
few-shot-image-classification-on-imagenet-5MAWS (ViT-2B)#3Top 1 Accuracy: 81.5
few-shot-image-classification-on-imagenet-5MAWS (ViT-H)#4Top 1 Accuracy: 79.8
image-classification-on-imagenetMAWS (ViT-6.5B)#8Top 1 Accuracy: 90.1%Number of params: 6500M
image-classification-on-imagenetMAWS (ViT-2B)#12Top 1 Accuracy: 89.8%Number of params: 2000M
image-classification-on-imagenetMAWS (ViT-H)#18Top 1 Accuracy: 89.5%Number of params: 650M
image-classification-on-imagenetMAWS (ViT-L)#30Top 1 Accuracy: 88.8%
image-classification-on-imagenetMAWS (ViT-B)#121Top 1 Accuracy: 86.8%
image-classification-on-imagenet-realMAWS (ViT-6.5B)#5Accuracy: 91.1%
image-classification-on-imagenet-realMAWS (ViT-2B)#10Accuracy: 90.9%
image-classification-on-imagenet-realMAWS (ViT-H)#12Accuracy: 90.8%
image-classification-on-imagenet-v2MAWS (ViT-6.5B)#5Top 1 Accuracy: 84.0
image-classification-on-imagenet-v2MAWS (ViT-2B)#7Top 1 Accuracy: 83.0
image-classification-on-inaturalist-2018MAWS (ViT-2B)#2Top-1 Accuracy: 91.3%
image-classification-on-objectnetMAWS (ViT-6.5B)#6Top-1 Accuracy: 77.9
image-classification-on-objectnetMAWS (ViT-2B)#7Top-1 Accuracy: 75.8
image-classification-on-objectnetMAWS (ViT-H)#8Top-1 Accuracy: 72.6
zero-shot-transfer-image-classification-on-1MAWS (ViT-2B)#8Accuracy (Private): 82.1
zero-shot-transfer-image-classification-on-1MAWS (ViT-H)#10Accuracy (Private): 81.1
zero-shot-transfer-image-classification-on-17MAWS (ViT-2B)#1Top 1 Accuracy: 96.2

Rank counts only results with a code link.