UltraBERT: Beyond the Limit of BERT Pre-Training
Open paper
Benchmark
Model
Rank
Results
natural-language-understanding-on-glue-dev
UltraBERT 380M
–
GLUE (dev): 91.0
question-answering-on-squad20-dev
UltraBERT 380M
–
F1: 90.9
Rank counts only results with a code link.