Why Does Self-Distillation (Sometimes) Degrade the Reasoning Capability of LLMs?

Benchmark Model Rank Results
mathematical-reasoning-on-math-500DeepSeek-R1-Distill-Qwen-7B (SFT on D_ug)#56Accuracy: 90.93
mathematical-reasoning-on-aime24DeepSeek-R1-Distill-Qwen-7B (SFT on D_ug)#79Acc: 51.04
mathematical-reasoning-on-aime-2025DeepSeek-R1-Distill-Qwen-7B (SFT on D_ug)#98Accuracy: 40.0