Rethinking Selective Knowledge Distillation
Open paper
Benchmark
Model
Rank
Results
language-modelling-on-lambada
SE-KD
#71
Perplexity: 6.9
arithmetic-reasoning-on-gsm8k
SE-KD + TopSmp (on-policy)
#197
Accuracy: 71.2
Rank counts only results with a code link.