Influence-Directed Distillation: Solving the Diversity Bottleneck in Sampled-Token On-Policy Distillation

Benchmark Model Rank Results
mathematical-reasoning-on-aime-2025Qwen3-4B-Non-Thinking + IDA-OPD–Accuracy: 53.3
mathematical-reasoning-on-aime24Qwen3-8B-Non-Thinking + IDA-OPD–Acc: 63.3
mathematical-reasoning-on-hmmt-feb-2025Qwen3-8B-Non-Thinking + IDA-OPD–Accuracy: 37.6