Cloze-driven Pretraining of Self-attention Networks

Benchmark Model Rank Results
constituency-parsing-on-penn-treebankCNN Large + fine-tune–F1 score: 95.6
named-entity-recognition-ner-on-conll-2003-englishCNN Large + fine-tune–F1: 93.5
sentiment-analysis-on-sst-2-binary-classificationCNN Large–Accuracy: 94.6