Post-Trained MoE Can Skip Half Experts via Self-Distillation

Benchmark Model Rank Results
code-generation-on-livecodebench-v5ZEDA (Qwen3-30B-A3B)#12Pass@1: 58.2
arithmetic-reasoning-on-gsm8kZEDA (Qwen3-30B-A3B)#18Accuracy: 95.5
mathematical-reasoning-on-math-500ZEDA (Qwen3-30B-A3B)#26Accuracy: 95.2
mathematical-reasoning-on-aime24ZEDA (GLM-4.7-Flash)#32Acc: 79.8
mathematical-reasoning-on-aime-2026ZEDA (GLM-4.7-Flash)#34Accuracy: 74.4
instruction-following-on-ifbenchZEDA (GLM-4.7-Flash)#38Accuracy: 47.3
code-generation-on-livecodebench-v6ZEDA (Qwen3-30B-A3B)#48Pass@1: 53.2