A Large Batch Optimizer Reality Check: Traditional, Generic Optimizers Suffice Across Batch Sizes
Open paper
Benchmark
Model
Rank
Results
image-classification-on-imagenet
ResNet-50 MLPerf v0.7 - 2512 steps
–
Top 1 Accuracy: 75.92%
question-answering-on-squad11
BERT-Large 32k batch size with AdamW
–
F1: 91.58
Rank counts only results with a code link.