TinyViT: Fast Pretraining Distillation for Small Vision Transformers

Open paper
Benchmark Model Rank Results
image-classification-on-imagenetTinyViT-21M-512-distill (512 res, 21k)#136Top 1 Accuracy: 86.5%Number of params: 21MGFLOPs: 27.0
image-classification-on-imagenetTinyViT-21M-384-distill (384 res, 21k)#161Top 1 Accuracy: 86.2%Number of params: 21MGFLOPs: 13.8
image-classification-on-imagenetTinyViT-21M-distill (21k)#283Top 1 Accuracy: 84.8%Number of params: 21MGFLOPs: 4.3
image-classification-on-imagenetTinyViT-11M-distill (21k)#446Top 1 Accuracy: 83.2%Number of params: 11MGFLOPs: 2.0
image-classification-on-imagenetTinyViT-21M#455Top 1 Accuracy: 83.1%Number of params: 21MGFLOPs: 4.3
image-classification-on-imagenetTinyViT-11M#610Top 1 Accuracy: 81.5%Number of params: 11MGFLOPs: 2.0
image-classification-on-imagenetTinyViT-5M-distill (21k)#654Top 1 Accuracy: 80.7%Number of params: 5.4MGFLOPs: 1.3
image-classification-on-imagenetTinyViT-5M#738Top 1 Accuracy: 79.1%Number of params: 5.4MGFLOPs: 1.3

Rank counts only results with a code link.