torchdistill: A Modular, Configuration-Driven Framework for Knowledge Distillation

Open paper
Benchmark Model Rank Results
image-classification-on-imagenetResNet-18 (PAD-L2 w/ ResNet-34 teacher)#942Top 1 Accuracy: 71.71%
image-classification-on-imagenetResNet-18 (FT w/ ResNet-34 teacher)#945Top 1 Accuracy: 71.56%
image-classification-on-imagenetResNet-18 (KD w/ ResNet-34 teacher)#948Top 1 Accuracy: 71.37%
image-classification-on-imagenetResNet-18 (L2 w/ ResNet-34 teacher)#950Top 1 Accuracy: 71.08%
image-classification-on-imagenetResNet-18 (CRD w/ ResNet-34 teacher)#952Top 1 Accuracy: 70.93%
image-classification-on-imagenetResNet-18 (tf-KD w/ ResNet-18 teacher)#955Top 1 Accuracy: 70.52%
image-classification-on-imagenetResNet-18 (SSKD w/ ResNet-34 teacher)#959Top 1 Accuracy: 70.09%
instance-segmentation-on-cocoMask R-CNN (Bottleneck-injected ResNet-50, FPN)#96mask AP: 33.6
object-detection-on-cocoMask R-CNN (Bottleneck-injected ResNet-50, FPN)#215box mAP: 36.9
object-detection-on-cocoFaster R-CNN (Bottleneck-injected ResNet-50 and FPN)#220box mAP: 35.9

Rank counts only results with a code link.