$V_kD:$ Improving Knowledge Distillation using Orthogonal Projections

Benchmark Model Rank Results
knowledge-distillation-on-imagenetVkD (T:RegNety 160 S:DeiT-S)#6Top-1 accuracy %: 82.9model size: 22MCRD training setting: ✘
knowledge-distillation-on-imagenetVkD (T:RegNety 160 S:DeiT-Ti)#15Top-1 accuracy %: 79.2model size: 6MCRD training setting: ✘