Rethinking Soft Labels for Knowledge Distillation: A Bias-Variance Tradeoff Perspective
Open paper
Benchmark
Model
Rank
Results
knowledge-distillation-on-imagenet
WSL (T: ResNet-34 S:ResNet-18)
#38
Top-1 accuracy %: 72.04
Rank counts only results with a code link.