Mesh-TensorFlow: Deep Learning for Supercomputers

Open paper
Benchmark Model Rank Results
language-modelling-on-one-billion-wordMesh Tensorflow#12PPL: 24.0Number of params: 4.9B

Rank counts only results with a code link.