Does On-Policy Distillation Really Distill? From Noisy Teacher to Self-Improvement

Benchmark Model Rank Results
mathematical-reasoning-on-hmmt-feb-2025OPSA (Qwen3.5-9B)#9Accuracy: 67.4
mathematical-reasoning-on-aime24OPSA (Qwen3.5-9B)#18Acc: 87.81
multiple-choice-question-answering-mcqa-on-gpqa-diamondOPSA (Qwen3.5-9B)#74Accuracy: 73.7