Cost-effective Selection of Pretraining Data: A Case Study of Pretraining BERT on Social Media

Benchmark Model Rank Results
clinical-concept-extraction-on-2010-i2b2vaClinicalBERT–Exact Span F1: 87.4