Knowledge Distillation from A Stronger Teacher

Benchmark Model Rank Results
knowledge-distillation-on-imagenetDIST (T: Swin-L S: Swin-T)#10Top-1 accuracy %: 82.3model size: 29MCRD training setting: ✘
knowledge-distillation-on-imagenetDIST (T: ResNet-34 S:ResNet-18)#36Top-1 accuracy %: 72.07CRD training setting: ✘
knowledge-distillation-on-cifar-100resnet8x4 (T: resnet32x4 S: resnet8x4)#13Top-1 Accuracy (%): 76.31