Toward Efficient Language Model Pretraining and Downstream Adaptation via Self-Evolution: A Case Study on SuperGLUE

Benchmark Model Rank Results
common-sense-reasoning-on-recordTuring NLR v5 XXL 5.4B (fine-tuned)EM: 95.9F1: 96.4
common-sense-reasoning-on-recordVega v2 6B (fine-tuned)EM: 93.9F1: 94.4
coreference-resolution-on-winograd-schemaTuring NLR v5 XXL 5.4B (fine-tuned)Accuracy: 97.3
coreference-resolution-on-winograd-schemaVega v2 6B (KD-based prompt transfer)Accuracy: 98.6
natural-language-inference-on-commitmentbankTuring NLR v5 XXL 5.4B (fine-tuned)Accuracy: 97.6F1: 95.9
natural-language-inference-on-commitmentbankVega v2 6B (KD-based prompt transfer)Accuracy: 99.2F1: 98.6
natural-language-inference-on-rteTuring NLR v5 XXL 5.4B (fine-tuned)Accuracy: 94.1%
natural-language-inference-on-rteVega v2 6B (KD-based prompt transfer)Accuracy: 96%
question-answering-on-boolqTuring NLR v5 XXL 5.4B (fine-tuned)Accuracy: 92
question-answering-on-boolqVega v2 6B (fine-tuned)Accuracy: 90.5
question-answering-on-copaTuring NLR v5 XXL 5.4B (fine-tuned)Accuracy: 98.2
question-answering-on-copaVega v2 6B (KD-based prompt transfer)Accuracy: 99.4
question-answering-on-multircTuring NLR v5 XXL 5.4B (fine-tuned)F1: 88.4EM: 63
question-answering-on-multircVega v2 6B (fine-tuned)F1: 88.2EM: 62.4
word-sense-disambiguation-on-words-in-contextTuring NLR v5 XXL 5.4B (fine-tuned)Accuracy: 77.1
word-sense-disambiguation-on-words-in-contextVega v2 6B (fine-tuned)Accuracy: 77.4