Exploring the Benefits of Training Expert Language Models over Instruction Tuning

Benchmark Model Rank Results
question-answering-on-storyclozeRoE-3B#8Accuracy: 86.33
natural-language-inference-on-anli-testRoE-3B#12A1: 35.49A2: 34.64A3: 31.22
word-sense-disambiguation-on-words-in-contextRoE-3B#15Accuracy: 52.97
question-answering-on-copaRoE-3B#34Accuracy: 79.25
coreference-resolution-on-winograd-schema-challengeRoE-3B#43Accuracy: 62.21
natural-language-inference-on-rteRoE-3B#62Accuracy: 64.01
common-sense-reasoning-on-winograndeRoE-3B#106Accuracy: 61.60
sentence-completion-on-hellaswagRoE-3B#210Accuracy: 34.6