An Empirical Study of Training Self-Supervised Vision Transformers

Open paper
Benchmark Model Rank Results
self-supervised-image-classification-on-imagenetMoCo v3 (ViT-BN-L/7)#18Top 1 Accuracy: 81.0%Number of Params: 304M
self-supervised-image-classification-on-imagenetMoCo v3 (ViT-BN-H)#34Top 1 Accuracy: 79.1%Number of Params: 700M
self-supervised-image-classification-on-imagenetMoCo v3 (ViT-H)#43Top 1 Accuracy: 78.1%Number of Params: 632M
self-supervised-image-classification-on-imagenetMoCo v3 (ViT-L)#47Top 1 Accuracy: 77.6%Number of Params: 307M
self-supervised-image-classification-on-imagenetMoCo v3 (ViT-B/16)#53Top 1 Accuracy: 76.7%Number of Params: 86M
self-supervised-image-classification-on-imagenet-finetunedMoCo v3 (ViT-L/16)#35Top 1 Accuracy: 84.1%Number of Params: 304M
self-supervised-image-classification-on-imagenet-finetunedMoCo v3 (ViT-B/16)#44Top 1 Accuracy: 83.2%Number of Params: 86M

Rank counts only results with a code link.