ModuleFormer: Modularity Emerges from Mixture-of-Experts

Benchmark Model Rank Results
zero-shot-question-answering-on-openbookqaMoLM-8B-K2 (0-shot)#25Accuracy: 23.8
common-sense-reasoning-on-arc-easyMoLM-8B-K2 (0-shot)#75Accuracy: 62.46
question-answering-on-piqaMoLM-8B-K2 (0-shot)#126Accuracy: 72.91
code-generation-on-humanevalMoLM-8B-K2#139Pass@1: 5.51
sentence-completion-on-hellaswagMoLM-8B-K2 (0-shot)#193Accuracy: 43.33