Knowledge Distillation Based on Transformed Teacher Matching

Benchmark Model Rank Results
knowledge-distillation-on-imagenetWTTM (T: DeiT III-Small S:DeiT-Tiny)#23Top-1 accuracy %: 77.03CRD training setting: ✘
knowledge-distillation-on-imagenetWTTM (T:resnet50, S:mobilenet-v1)#26Top-1 accuracy %: 73.09
knowledge-distillation-on-imagenetWTTM (T: ResNet-34 S:ResNet-18)#35Top-1 accuracy %: 72.19CRD training setting: ✓