ScaleKD: Strong Vision Transformers Could Be Excellent Teachers

Open paper
Benchmark Model Rank Results
knowledge-distillation-on-imagenetScaleKD (T:BEiT-L S:ViT-B/14)#1Top-1 accuracy %: 86.43model size: 87MCRD training setting: ✘
knowledge-distillation-on-imagenetScaleKD (T:Swin-L S:ViT-B/16)#2Top-1 accuracy %: 85.53model size: 87MCRD training setting: ✘
knowledge-distillation-on-imagenetScaleKD (T:Swin-L S:ViT-S/16)#3Top-1 accuracy %: 83.93model size: 22MCRD training setting: ✘
knowledge-distillation-on-imagenetScaleKD (T:Swin-L S:Swin-T)#4Top-1 accuracy %: 83.8model size: 27MCRD training setting: ✘
knowledge-distillation-on-imagenetScaleKD (T:Swin-L S:ResNet-50)#8Top-1 accuracy %: 82.55model size: 22MCRD training setting: ✘
image-classification-on-imagenetViT-B/14#152Top 1 Accuracy: 86.43%

Rank counts only results with a code link.