Large Batch Optimization for Deep Learning: Training BERT in 76 minutes

Open paper

Benchmark	Model	Rank	Results
question-answering-on-squad11-dev	BERT large (LAMB optimizer)	#43	F1: 90.584