torchdistill: A Modular, Configuration-Driven Framework for Knowledge Distillation

Open paper
Benchmark Model Rank Results
instance-segmentation-on-coco-test-devMask R-CNN (Bottleneck-injected ResNet-50, FPN)#98mask AP: 33.6
object-detection-on-coco-val2017Mask R-CNN (Bottleneck-injected ResNet-50, FPN)#429box AP: 36.9
object-detection-on-coco-val2017Faster R-CNN (Bottleneck-injected ResNet-50 and FPN)#433box AP: 35.9
image-classification-on-imagenetResNet-18 (PAD-L2 w/ ResNet-34 teacher)#1056Top 1 Accuracy: 71.71%
image-classification-on-imagenetResNet-18 (FT w/ ResNet-34 teacher)#1059Top 1 Accuracy: 71.56%
image-classification-on-imagenetResNet-18 (KD w/ ResNet-34 teacher)#1063Top 1 Accuracy: 71.37%
image-classification-on-imagenetResNet-18 (L2 w/ ResNet-34 teacher)#1067Top 1 Accuracy: 71.08%
image-classification-on-imagenetResNet-18 (CRD w/ ResNet-34 teacher)#1069Top 1 Accuracy: 70.93%
image-classification-on-imagenetResNet-18 (tf-KD w/ ResNet-18 teacher)#1072Top 1 Accuracy: 70.52%
image-classification-on-imagenetResNet-18 (SSKD w/ ResNet-34 teacher)#1076Top 1 Accuracy: 70.09%

Rank counts only results with a code link.