Self-Attention with Relative Position Representations

Benchmark Model Rank Results
machine-translation-on-wmt2014-english-frenchTransformer (big) + Relative Position Representations#22BLEU score: 41.5
machine-translation-on-wmt2014-english-germanTransformer (big) + Relative Position Representations#26BLEU score: 29.2