Coupling Experts and Routers in Mixture-of-Experts via an Auxiliary Loss
Open paper
Benchmark
Model
Rank
Results
arithmetic-reasoning-on-gsm8k
MoE + ERC
–
Accuracy: 45.8
math-word-problem-solving-on-math
MoE + ERC
–
Accuracy: 26.1
multi-task-language-understanding-on-mmlu-pro
MoE + ERC
–
Accuracy: 31.9
Rank counts only results with a code link.