MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devices

Benchmark Model Rank Results
natural-language-understanding-on-glue-testMobileBERT w/o OPT#8GLUE (test): 78.5
question-answering-on-squad20-devMobileBERT w/o OPT#15F1: 80.2
semantic-textual-similarity-on-mrpcMobileBERT#15Accuracy: 88.8%
natural-language-inference-on-qnliMobileBERT w/o OPT#27Accuracy: 91.6%
paraphrase-identification-on-quora-question-pairsMobileBERT w/o OPT#28F1: 70.5
linguistic-acceptability-on-colaMobileBERT w/o OPT#33Accuracy: 51.1%
sentiment-analysis-on-sst-2-binary-classificationMobileBERT#44Accuracy: 92.8
natural-language-inference-on-rteMobileBERT w/o OPT#49Accuracy: 70.4%
question-answering-on-squad11-devMobileBERT w/o OPT#53F1: 90.3