Very Deep Transformers for Neural Machine Translation

Benchmark Model Rank Results
machine-translation-on-wmt2014-english-frenchTransformer+BT (ADMIN init)#1BLEU score: 46.4SacreBLEU: 44.4
machine-translation-on-wmt2014-english-frenchTransformer (ADMIN init)#6BLEU score: 43.8SacreBLEU: 41.8
machine-translation-on-wmt2014-english-germanTransformer (ADMIN init)#13BLEU score: 30.1SacreBLEU: 29.5Number of Params: 256M