Cross-lingual Language Model Pretraining

Benchmark Model Rank Results
machine-translation-on-wmt2016-romanian-englishMLM pretraining#4BLEU score: 35.3
unsupervised-machine-translation-on-wmt2014-english-frenchMLM pretraining for encoder and decoder#4BLEU: 33.4
unsupervised-machine-translation-on-wmt2014-french-englishMLM pretraining for encoder and decoder#4BLEU: 33.3
unsupervised-machine-translation-on-wmt2016-english-germanMLM pretraining for encoder and decoder#4BLEU: 26.4
unsupervised-machine-translation-on-wmt2016-german-englishMLM pretraining for encoder and decoder#4BLEU: 34.3