TinyViT: Fast Pretraining Distillation for Small Vision Transformers

Open paper
Benchmark Model Rank Results
image-classification-on-imagenetTinyViT-21M-512-distill (512 res, 21k)#148Top 1 Accuracy: 86.5%Number of params: 21MGFLOPs: 27.0
image-classification-on-imagenetTinyViT-21M-384-distill (384 res, 21k)#178Top 1 Accuracy: 86.2%Number of params: 21MGFLOPs: 13.8
image-classification-on-imagenetTinyViT-21M-distill (21k)#309Top 1 Accuracy: 84.8%Number of params: 21MGFLOPs: 4.3
image-classification-on-imagenetTinyViT-11M-distill (21k)#490Top 1 Accuracy: 83.2%Number of params: 11MGFLOPs: 2.0
image-classification-on-imagenetTinyViT-21M#499Top 1 Accuracy: 83.1%Number of params: 21MGFLOPs: 4.3
image-classification-on-imagenetTinyViT-11M#670Top 1 Accuracy: 81.5%Number of params: 11MGFLOPs: 2.0
image-classification-on-imagenetTinyViT-5M-distill (21k)#725Top 1 Accuracy: 80.7%Number of params: 5.4MGFLOPs: 1.3
image-classification-on-imagenetTinyViT-5M#819Top 1 Accuracy: 79.1%Number of params: 5.4MGFLOPs: 1.3

Rank counts only results with a code link.