Mixtral of Experts

Benchmark Model Rank Results
code-generation-on-mbppMixtral 8x7B (3-shot)#37Accuracy: 60.7
common-sense-reasoning-on-arc-easyMixtral 8x7B (0-shot)#10Accuracy: 83.1
common-sense-reasoning-on-arc-easyMistral 7B (0-shot)#12Accuracy: 80.5
common-sense-reasoning-on-winograndeMixtral 8x7B (0-shot)#19Accuracy: 77.2
common-sense-reasoning-on-winograndeMistral 7B (0-shot)#27Accuracy: 74.2
math-word-problem-solving-on-mathMixtral 8x7B (maj@4)#79Accuracy: 28.4
math-word-problem-solving-on-mathMistral 7B (maj@4)#99Accuracy: 12.7Parameters (Billions): 7
multi-task-language-understanding-on-mmluMixtral 8x7B (5-shot)#6Average (%): 70.6
multi-task-language-understanding-on-mmluMistral 7B (5-shot)#12Average (%): 62.5
question-answering-on-piqaMixtral 8x7B (0-shot)#12Accuracy: 83.6
question-answering-on-piqaMistral 7B (0-shot)#21Accuracy: 82.2