UniPool: A Globally Shared Expert Pool for Mixture-of-Experts

Benchmark Model Rank Results
reading-comprehension-on-raceUniPool (978M)#33Accuracy: 29.38
language-modelling-on-lambadaUniPool (978M)#57Accuracy: 44.94
common-sense-reasoning-on-arc-easyUniPool (978M)#84Accuracy: 54.34
question-answering-on-piqaUniPool (978M)#157Accuracy: 69.21
common-sense-reasoning-on-winograndeUniPool (978M)#161Accuracy: 52.17
sentence-completion-on-hellaswagUniPool (978M)#208Accuracy: 36.19