| Understanding the Role of the Projector in Knowledge Distillation | ✓ Link | 79.86 | SRD (T:resnet-32x4, S:shufflenet-v2) | 2023-03-20 |
| Logit Standardization in Knowledge Distillation | ✓ Link | 78.76 | shufflenet-v2(T:resnet-32x4, S:shufflenet-v2) | 2024-03-03 |
| MV-MR: multi-views and multi-representations for self-supervised learning and knowledge distillation | ✓ Link | 78.6 | MV-MR (T: CLIP/ViT-B-16 S: resnet50) | 2023-03-21 |
| Logit Standardization in Knowledge Distillation | ✓ Link | 78.28 | resnet8x4
(T: resnet32x4 S: resnet8x4) | 2024-03-03 |
| Knowledge Distillation with the Reused Teacher Classifier | ✓ Link | 78.08 | resnet8x4 (T: resnet32x4 S: resnet8x4 [modified]) | 2022-03-26 |
| Improving Knowledge Distillation via Regularizing Feature Norm and Direction | ✓ Link | 77.93 | ReviewKD++(T:resnet-32x4, S:shufflenet-v2) | 2023-05-26 |
| Improving Knowledge Distillation via Regularizing Feature Norm and Direction | ✓ Link | 77.68 | ReviewKD++(T:resnet-32x4, S:shufflenet-v1) | 2023-05-26 |
| LumiNet: The Bright Side of Perceptual Knowledge Distillation | ✓ Link | 77.50 | resnet8x4 (T: resnet32x4 S: resnet8x4) | 2023-10-05 |
| Information Theoretic Representation Distillation | ✓ Link | 76.68 | resnet8x4 (T: resnet32x4 S: resnet8x4) | 2021-12-01 |
| Knowledge Distillation from A Stronger Teacher | ✓ Link | 76.31 | resnet8x4 (T: resnet32x4 S: resnet8x4) | 2022-05-21 |
| Improving Knowledge Distillation via Regularizing Feature Norm and Direction | ✓ Link | 76.28 | DKD++(T:resnet-32x4, S:resnet-8x4) | 2023-05-26 |
| Wasserstein Contrastive Representation Distillation | | 76.15 | resnet8x4 (T: resnet32x4 S: resnet8x4) | 2020-12-15 |
| Improving Knowledge Distillation via Regularizing Feature Norm and Direction | ✓ Link | 75.66 | ReviewKD++(T:WRN-40-2, S:WRN-40-1) | 2023-05-26 |
| Distilling Knowledge via Knowledge Review | ✓ Link | 75.63 | resnet8x4 (T: resnet32x4 S: resnet8x4) | 2021-04-19 |
| Contrastive Representation Distillation | ✓ Link | 75.51 | resnet8x4 (T: resnet32x4 S: resnet8x4) | 2019-10-23 |
| Information Theoretic Representation Distillation | ✓ Link | 74.93 | vgg8 (T:vgg13 S:vgg8) | 2021-12-01 |
| Distilling Knowledge via Knowledge Review | ✓ Link | 74.84 | vgg8 (T:vgg13 S:vgg8) | 2021-04-19 |
| Wasserstein Contrastive Representation Distillation | | 74.72 | vgg8 (T:vgg13 S:vgg8) | 2020-12-15 |
| Contrastive Representation Distillation | ✓ Link | 74.29 | vgg8 (T:vgg13 S:vgg8) | 2019-10-23 |
| Distilling the Knowledge in a Neural Network | ✓ Link | 73.33 | resnet8x4 (T: resnet32x4 S: resnet8x4) | 2015-03-09 |
| Distilling the Knowledge in a Neural Network | ✓ Link | 72.98 | vgg8 (T:vgg13 S:vgg8) | 2015-03-09 |
| Improving Knowledge Distillation via Regularizing Feature Norm and Direction | ✓ Link | 72.53 | KD++(T:resnet56, S:resnet20) | 2023-05-26 |
| Information Theoretic Representation Distillation | ✓ Link | 71.99 | resnet110 (T:resnet110 S:resnet20) | 2021-12-01 |
| Wasserstein Contrastive Representation Distillation | | 71.88 | resnet110 (T:resnet110 S:resnet20) | 2020-12-15 |
| Contrastive Representation Distillation | ✓ Link | 71.56 | resnet110 (T:resnet110 S:resnet20) | 2019-10-23 |
| Improving Knowledge Distillation via Regularizing Feature Norm and Direction | ✓ Link | 70.82 | DKD++(T:resnet50, S:mobilenetv2) | 2023-05-26 |
| Distilling the Knowledge in a Neural Network | ✓ Link | 70.67 | resnet110 (T:resnet110 S:resnet20) | 2015-03-09 |