Deep contextualized word representations

Benchmark Model Rank Results
citation-intent-classification-on-acl-arcBiLSTM-Attention + ELMo#4Macro-F1: 54.6
coreference-resolution-on-ontonotese2e-coref + ELMo#19F1: 70.4
named-entity-recognition-ner-on-conll-2003BiLSTM-CRF+ELMo#43F1: 92.22
named-entity-recognition-on-conllBiLSTM-CRF+ELMo#9F1: 93.42
natural-language-inference-on-snliESIM + ELMo Ensemble#14% Test Accuracy: 89.3% Train Accuracy: 92.1Parameters: 40m
natural-language-inference-on-snliESIM + ELMo#20% Test Accuracy: 88.7% Train Accuracy: 91.6Parameters: 8.0m
question-answering-on-squad11BiDAF + Self Attention + ELMo (ensemble)#11EM: 81.003F1: 87.432
question-answering-on-squad11BiDAF + Self Attention + ELMo (single model)#16EM: 78.58F1: 85.833
question-answering-on-squad11-devBiDAF + Self Attention + ELMo#45F1: 85.6
question-answering-on-squad20BiDAF + Self Attention + ELMo (single model)#23EM: 63.372F1: 66.251
semantic-role-labeling-on-ontonotesHe et al., 2017 + ELMo#13F1: 84.6
sentiment-analysis-on-sst-5-fine-grainedBCN+ELMo#7Accuracy: 54.7
task-1-grouping-on-ocwELMo (LARGE)#20# Correct Groups: 55 ± 4Fowlkes Mallows Score (FMS): 29.5 ± .3
word-sense-disambiguation-on-supervisedELMo#16Senseval 2: 71.6Senseval 3: 69.6SemEval 2007: 62.2