Composing What Each Teacher Learned: Multi-Teacher On-Policy Distillation through Teacher-Relative Shifts

Open paper

Benchmark	Model	Rank	Results
mathematical-reasoning-on-aime-2025	Δ-MOPD (1.5B student, Avg@16)	–	Accuracy: 32.92
mathematical-reasoning-on-math-500	Δ-MOPD (1.5B student)	–	Accuracy: 59.4