PeCo: Perceptual Codebook for BERT Pre-training of Vision Transformers

Open paper
Benchmark Model Rank Results
image-classification-on-imagenetPeCo (ViT-H, 448)#53Top 1 Accuracy: 88.3%
image-classification-on-imagenetPeCo (ViT-H, 224)#81Top 1 Accuracy: 87.5%
self-supervised-image-classification-on-1PeCo(ViT-H/14, 448)#4Top 1 Accuracy: 88.3%Number of Params: 632M

Rank counts only results with a code link.