Weight Averaging Improves Knowledge Distillation under Domain Shift

Benchmark Model Rank Results
domain-generalization-on-office-homeWAKD (DeiT-Ti)#29Average Accuracy: 70.5
domain-generalization-on-office-homeWAKD (Resnet-18)#38Average Accuracy: 66.7
domain-generalization-on-pacsWAKD (DeiT-Ti)#34Average Accuracy: 87.6
domain-generalization-on-pacsWAKD (Resnet-18)#42Average Accuracy: 86.6