WinoGrande: An Adversarial Winograd Schema Challenge at Scale

Benchmark Model Rank Results
common-sense-reasoning-on-winograndeRoBERTa-Winogrande 355M (fine-tuned)#17Accuracy: 79.1
common-sense-reasoning-on-winograndeBERT-Winogrande 345M (fine-tuned)#41Accuracy: 64.9
common-sense-reasoning-on-winograndeRoBERTa-DPR 355M (0-shot)#49Accuracy: 58.9
common-sense-reasoning-on-winograndeBERT-large 345M (0-shot)#58Accuracy: 51.9
common-sense-reasoning-on-winograndeBERT-DPR 345M (0-shot)#59Accuracy: 51
common-sense-reasoning-on-winograndeRoBERTa-large 355M (0-shot)#60Accuracy: 50
coreference-resolution-on-winograd-schemaRoBERTa-WinoGrande 355M#7Accuracy: 90.1
coreference-resolution-on-winograd-schemaRoBERTa-DPR 355M#16Accuracy: 83.1
coreference-resolution-on-winograd-schemaWKH#53Accuracy: 57.1
coreference-resolution-on-winograd-schemaKEE+NKAM on WinoGrande#58Accuracy: 52.8
question-answering-on-copaRoBERTa-Winogrande-ft 355M (fine-tuned)#15Accuracy: 90.6
question-answering-on-copaRoBERTa-ft 355M (fine-tuned)#22Accuracy: 86.4
question-answering-on-copaRoBERTa-Winogrande 355M (fine-tuned)#27Accuracy: 84.4
question-answering-on-copaCausal Strength w/multi-word predicates (presumably on WinoGrande?)#39Accuracy: 76.4
question-answering-on-copaPointwise Mutual Information (on 10M stories)#44Accuracy: 65.4