XLNet: Generalized Autoregressive Pretraining for Language Understanding

Benchmark Model Rank Results
linguistic-acceptability-on-colaXLNet (single model)#10Accuracy: 69%
natural-language-inference-on-anli-testXLNet (Large)#5A1: 70.3A2: 50.9A3: 49.4
natural-language-inference-on-multinliXLNet (single model)#6Matched: 90.8
natural-language-inference-on-qnliXLNet (single model)#9Accuracy: 94.9%
natural-language-inference-on-rteXLNet (single model)#21Accuracy: 85.9%
natural-language-inference-on-wnliXLNet#3Accuracy: 92.5
paraphrase-identification-on-quora-questionXLNet-Large (ensemble)#6F1: 74.2Accuracy: 90.3
question-answering-on-quora-question-pairsXLNet (single model)#1Accuracy: 92.3%
question-answering-on-raceXLNet#1RACE-m: 85.45RACE: 81.75
question-answering-on-squad11XLNet (single model)#3EM: 89.898F1: 95.080Hardware Burden: 46449G
question-answering-on-squad11-devXLNet (single model)#4EM: 89.7F1: 95.1
question-answering-on-squad20XLNet (single model)#9EM: 87.926F1: 90.689
question-answering-on-squad20-devXLNet (single model)#1F1: 90.6EM: 87.9
reading-comprehension-on-raceXLNet#7Accuracy (Middle): 88.6Accuracy (High): 84.0
semantic-textual-similarity-on-mrpcXLNet (single model)#8Accuracy: 90.8%
semantic-textual-similarity-on-sts-benchmarkXLNet (single model)#2Pearson Correlation: 0.925
sentiment-analysis-on-imdbXLNet#3Accuracy: 96.21
sentiment-analysis-on-sst-2-binaryXLNet (single model)#6Accuracy: 97
sentiment-analysis-on-sst-2-binaryXLNet-Large (ensemble)#9Accuracy: 96.8
sentiment-analysis-on-yelp-binaryXLNet#1Error: 1.37
sentiment-analysis-on-yelp-fine-grainedXLNet#1Error: 27.05
text-classification-on-ag-newsXLNet#1Error: 4.45
text-classification-on-dbpediaXLNet#1Error: 0.62
text-classification-on-yelp-2XLNet#1Accuracy: 98.63%
text-classification-on-yelp-5XLNet#2Accuracy: 72.95%