Branch-Train-MiX: Mixing Expert LLMs into a Mixture-of-Experts LLM

Benchmark Model Rank Results
arithmetic-reasoning-on-gsm8kBranch-Train-MiX 4x7B (sampling top-2 experts)#101Accuracy: 37.1
code-generation-on-mbppBranch-Train-Merge 4x7B (top-2)#69Accuracy: 42.6
code-generation-on-mbppBranch-Train-MiX 4x7B (sampling top-2 experts)#73Accuracy: 39.4
common-sense-reasoning-on-winograndeBranch-Train-MiX 4x7B (sampling top-1 expert)#34Accuracy: 70.6
math-word-problem-solving-on-mathBranch-Train-MiX 4x7B (sampling top-2 experts)#92Accuracy: 17.8
multi-task-language-understanding-on-mmluBranch-Train-MiX 4x7B (sampling top-1 experts)#17Average (%): 53.2
question-answering-on-triviaqaBranch-Train-MiX 4x7B (sampling top-2 experts)#23EM: 57.1