MASS: Masked Sequence to Sequence Pre-training for Language Generation

Benchmark Model Rank Results
unsupervised-machine-translation-on-wmt2014-english-frenchMASS (6-layer Transformer)#2BLEU: 37.5
unsupervised-machine-translation-on-wmt2014-french-englishMASS (6-layer Transformer)#2BLEU: 34.9
unsupervised-machine-translation-on-wmt2016-english-germanMASS (6-layer Transformer)#2BLEU: 28.3
unsupervised-machine-translation-on-wmt2016-german-englishMASS (6-layer Transformer)#2BLEU: 35.2
text-summarization-on-gigawordMASS#17ROUGE-1: 38.73ROUGE-2: 19.71ROUGE-L: 35.96