Exploring the Benefits of Training Expert Language Models over Instruction Tuning

Benchmark Model Rank Results
common-sense-reasoning-on-winograndeRoE-3B#44Accuracy: 61.60
coreference-resolution-on-winograd-schemaRoE-3B#43Accuracy: 62.21
natural-language-inference-on-anli-testRoE-3B#12A1: 35.49A2: 34.64A3: 31.22
natural-language-inference-on-rteRoE-3B#58Accuracy: 64.01
question-answering-on-copaRoE-3B#34Accuracy: 79.25
question-answering-on-storyclozeRoE-3B#8Accuracy: 86.33
sentence-completion-on-hellaswagRoE-3B#66Accuracy: 34.6
word-sense-disambiguation-on-words-in-contextRoE-3B#15Accuracy: 52.97