DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter

Benchmark Model Rank Results
linguistic-acceptability-on-colaDistilBERT 66M#30Accuracy: 49.1%
natural-language-inference-on-qnliDistilBERT 66M#30Accuracy: 90.2%
natural-language-inference-on-rteDistilBERT 66M#60Accuracy: 62.9%
natural-language-inference-on-wnliDistilBERT 66M#21Accuracy: 44.4
question-answering-on-multitqDistillBERT#8Hits@1: 8.3Hits@10: 48.4
question-answering-on-quora-question-pairsDistilBERT 66M#13Accuracy: 89.2%
question-answering-on-squad11-devDistilBERT#20EM: 77.7
question-answering-on-squad11-devDistilBERT 66M#44F1: 85.8
semantic-textual-similarity-on-mrpcDistilBERT 66M#10Accuracy: 90.2%
semantic-textual-similarity-on-sts-benchmarkDistilBERT 66M#11Pearson Correlation: 0.907
sentiment-analysis-on-imdbDistilBERT 66M#28Accuracy: 92.82
sentiment-analysis-on-sst-2-binaryDistilBERT 66M#50Accuracy: 91.3
task-1-grouping-on-ocwDistilBERT (BASE)#21# Correct Groups: 49 ± 4Fowlkes Mallows Score (FMS): 29.1 ± .2