The Evolved Transformer

Benchmark Model Rank Results
language-modelling-on-one-billion-wordEvolved Transformer Big#16PPL: 28.6
machine-translation-on-wmt2014-english-frenchEvolved Transformer Big#25BLEU score: 41.3
machine-translation-on-wmt2014-english-frenchEvolved Transformer Base#28BLEU score: 40.6
machine-translation-on-wmt2014-english-germanEvolved Transformer Big#16BLEU score: 29.8SacreBLEU: 29.2Number of Params: 218M
machine-translation-on-wmt2014-english-germanEvolved Transformer Base#39BLEU score: 28.4Hardware Burden: 2488G