BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Benchmark Model Rank Results
question-answering-on-coqaBERT Large Augmented (single model)#1In-domain: 82.5Out-of-domain: 77.6Overall: 81.1
question-answering-on-coqaBERT-base finetune (single model)#2In-domain: 79.8Out-of-domain: 74.1Overall: 78.1
natural-language-understanding-on-pdp60BERT-large 340M#2Accuracy: 78.3
text-classification-on-dbpediaBidirectional Encoder Representations from Transformers#2Error: 0.64
multimodal-intent-recognition-on-photochatBERT#4F1: 53.2Precision: 56.1Recall: 50.6
emotion-recognition-in-conversation-on-cpedBERT_{utt}#5Accuracy of Sentiment: 48.96Macro-F1 of Sentiment: 45.18
question-answering-on-squad11BERT (ensemble)#6EM: 87.433F1: 93.160
question-answering-on-squad11BERT-LARGE (Ensemble+TriviaQA)#7EM: 87.4F1: 93.2
question-answering-on-squad11BERT (single model)#8EM: 85.083F1: 91.835
question-answering-on-squad11BERT-LARGE (Single+TriviaQA)#49F1: 91.8
type-prediction-on-manytypes4typescriptBERT#6Average Accuracy: 57.52Average Precision: 54.18Average Recall: 54.02…
natural-language-understanding-on-glue-testBERT Large#7GLUE (test): 80.5
question-answering-on-squad11-devBERT-LARGE (Ensemble+TriviaQA)#7EM: 86.2F1: 92.2
question-answering-on-squad11-devBERT-LARGE (Single+TriviaQA)#9EM: 84.2F1: 91.1
question-answering-on-multitqBERT#11Hits@1: 8.3Hits@10: 48.2
common-sense-reasoning-on-recordBERT-Base (single model)#12EM: 54.040F1: 56.065
paraphrase-identification-on-quora-question-pairsBERT-LARGE#12F1: 72.1
question-answering-on-multircBERT-large(single model)#14F1: 70.0EM: 24.1
stock-market-prediction-on-astockBert Chinese#15Accuray: 59.11F1-score: 58.99Recall: 59.20Precision: 59.07
natural-language-inference-on-multinliBERT-LARGE#20Matched: 86.7Mismatched: 85.9
natural-language-inference-on-wnliBERT-large 340M#20Accuracy: 65.1
linguistic-acceptability-on-colaBERT-LARGE#24Accuracy: 60.5%
natural-language-inference-on-qnliBERT-LARGE#24Accuracy: 92.7%
question-answering-on-squad20BERT Large#27F1: 83.1
sentiment-analysis-on-sst-2-binary-classificationBERT-LARGE#28Accuracy: 94.9
semantic-textual-similarity-on-mrpcBERT-LARGE#33F1: 89.3
semantic-textual-similarity-on-sts-benchmarkBERT-LARGE#37Spearman Correlation: 0.865
coreference-resolution-on-winograd-schema-challengeBERT-large 340M#44Accuracy: 62.0
natural-language-inference-on-rteBERT-large 340M#50Accuracy: 70.1%
question-answering-on-piqaBERT-Large 340M#164Accuracy: 66.7