BERT, mBERT, or BiBERT? A Study on Contextualized Embeddings for Neural Machine Translation

Benchmark Model Rank Results
machine-translation-on-iwslt2014-germanBiBERT#1BLEU score: 38.61Number of Params: 73.8M
machine-translation-on-wmt2014-english-germanBiBERT#5BLEU score: 31.26
machine-translation-on-wmt2014-german-englishBiBERT#2BLEU score: 34.94