Improving Language Understanding by Generative Pre-Training

Benchmark Model Rank Results
question-answering-on-raceFinetuned Transformer LM#2RACE-m: 62.9RACE-h: 57.4RACE: 59.0
natural-language-inference-on-scitailFinetuned Transformer LM#3Accuracy: 88.3
question-answering-on-storyclozeFinetuned Transformer LM#7Accuracy: 86.5
natural-language-understanding-on-glue-testGPT-1#9GLUE (test): 72.8
natural-language-inference-on-snliFine-Tuned LM-Pretrained Transformer#11% Test Accuracy: 89.9% Train Accuracy: 96.6Parameters: 85m
reading-comprehension-on-raceGPT-1#26Accuracy: 59.0
paraphrase-identification-on-quora-question-pairsGPT-1#29F1: 70.3
natural-language-inference-on-multinliFinetuned Transformer LM#34Matched: 82.1Mismatched: 81.4
linguistic-acceptability-on-colaGPT-1#36Accuracy: 45.4%
semantic-textual-similarity-on-mrpcGPT-1#36F1: 82.3
natural-language-inference-on-qnliGPT-1#38Accuracy: 88.1%
sentiment-analysis-on-sst-2-binary-classificationGPT-1#53Accuracy: 91.3
natural-language-inference-on-rteGPT-1#77Accuracy: 56.0%