XLNet: Generalized Autoregressive Pretraining for Language Understanding

Benchmark Model Rank Results
question-answering-on-quora-question-pairsXLNet (single model)#1Accuracy: 92.3%
question-answering-on-raceXLNet#1RACE-m: 85.45RACE: 81.75
sentiment-analysis-on-yelp-binary-classificationXLNet#1Error: 1.37
sentiment-analysis-on-yelp-fine-grained-classificationXLNet#1Error: 27.05
text-classification-on-ag-newsXLNet#1Error: 4.45
text-classification-on-dbpediaXLNet#1Error: 0.62
text-classification-on-yelp-2XLNet#1Accuracy: 98.63%
semantic-textual-similarity-on-sts-benchmarkXLNet (single model)#2Pearson Correlation: 0.925
text-classification-on-yelp-5XLNet#2Accuracy: 72.95%
question-answering-on-squad11XLNet (single model)#3EM: 89.898F1: 95.080Hardware Burden: 46449G
question-answering-on-squad20-devXLNet (single model)#3F1: 90.6EM: 87.9
sentiment-analysis-on-imdbXLNet#3Accuracy: 96.21
natural-language-inference-on-wnliXLNet#4Accuracy: 92.5
question-answering-on-squad11-devXLNet (single model)#4EM: 89.7F1: 95.1
natural-language-inference-on-anli-testXLNet (Large)#5A1: 70.3A2: 50.9A3: 49.4
natural-language-inference-on-multinliXLNet (single model)#6Matched: 90.8
paraphrase-identification-on-quora-question-pairsXLNet-Large (ensemble)#7F1: 74.2Accuracy: 90.3
sentiment-analysis-on-sst-2-binary-classificationXLNet (single model)#7Accuracy: 97
sentiment-analysis-on-sst-2-binary-classificationXLNet-Large (ensemble)#10Accuracy: 96.8
semantic-textual-similarity-on-mrpcXLNet (single model)#8Accuracy: 90.8%
question-answering-on-squad20XLNet (single model)#9EM: 87.926F1: 90.689
natural-language-inference-on-qnliXLNet (single model)#11Accuracy: 94.9%
linguistic-acceptability-on-colaXLNet (single model)#12Accuracy: 69%
reading-comprehension-on-raceXLNet#12Accuracy (Middle): 88.6Accuracy (High): 84.0
natural-language-inference-on-rteXLNet (single model)#23Accuracy: 85.9%