LastOPD: Taming Collapse in Latent On-Policy Distillation

Benchmark Model Rank Results
mathematical-reasoning-on-minerva-mathLastOPD (Qwen3-4B -> Qwen3-1.7B-Base)#42Accuracy: 17.74
mathematical-reasoning-on-olympiadbenchLastOPD (Qwen3-4B -> Qwen3-1.7B-Base)#49Accuracy: 24.26
mathematical-reasoning-on-math-500LastOPD (Qwen3-4B -> Qwen3-1.7B-Base)#120Accuracy: 58.95
mathematical-reasoning-on-aime24LastOPD (Qwen3-8B -> Qwen3-1.7B-Base)#133Acc: 7.92
mathematical-reasoning-on-aime-2025LastOPD (Qwen3-8B -> Qwen3-1.7B-Base)#145Accuracy: 4.58
arithmetic-reasoning-on-gsm8kLastOPD (Qwen3-4B -> Qwen3-1.7B-Base)#173Accuracy: 75.82