PeCo: Perceptual Codebook for BERT Pre-training of Vision Transformers

Open paper
Benchmark Model Rank Results
self-supervised-image-classification-on-imagenet-finetunedPeCo(ViT-H/14, 448)#4Top 1 Accuracy: 88.3%Number of Params: 632M
image-classification-on-imagenetPeCo (ViT-H, 448)#59Top 1 Accuracy: 88.3%
image-classification-on-imagenetPeCo (ViT-H, 224)#88Top 1 Accuracy: 87.5%

Rank counts only results with a code link.