TIP: Token Importance in On-Policy Distillation

Benchmark Model Rank Results
mathematical-reasoning-on-math-500TIP Soft-OR 50% (Qwen3-8B โ†’ Qwen3-4B)#96Accuracy: 79.1
mathematical-reasoning-on-aime24TIP Soft-OR 50% (Llama-3.3-70B โ†’ Llama-3.1-8B)#108Acc: 26.0
mathematical-reasoning-on-aime-2025TIP Soft-OR 20% (Qwen3-8B โ†’ Qwen3-4B)#121Accuracy: 23.2