Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model
Open paper
Benchmark
Model
Rank
Results
sentence-completion-on-hellaswag
Megatron-Turing NLG 530B (Few-Shot)
#57
Accuracy: 82.4
sentence-completion-on-hellaswag
Megatron-Turing NLG 530B (1-shot)
#75
Accuracy: 80.2
Rank counts only results with a code link.