Task Compass: Scaling Multi-task Pre-training with Task Prefix

Benchmark Model Rank Results
common-sense-reasoning-on-winograndeCompassMTL 567M with Tailor#3Accuracy: 90.5
common-sense-reasoning-on-winograndeCompassMTL 567M#4Accuracy: 89.6
common-sense-reasoning-on-winograndeExDeBERTa 567M#7Accuracy: 87
question-answering-on-piqaCompassMTL 567M with Tailor#3Accuracy: 88.3
question-answering-on-piqaCompassMTL 567M#6Accuracy: 87.3
question-answering-on-piqaExDeBERTa 567M#9Accuracy: 85.5
question-answering-on-social-iqaCompassMTL 567M with Tailor#3Accuracy: 82.2
question-answering-on-social-iqaCompassMTL 567M#4Accuracy: 81.7
question-answering-on-social-iqaExDeBERTa 567M#9Accuracy: 79.6
sentence-completion-on-hellaswagCompassMTL 567M with Tailor#1Accuracy: 96.1
sentence-completion-on-hellaswagCompassMTL 567M#2Accuracy: 95.6
sentence-completion-on-hellaswagExDeBERTa 567M#24Accuracy: 83.6