RoFormer: Enhanced Transformer with Rotary Position Embedding

Benchmark Model Rank Results
paraphrase-identification-on-mrpc-glue-devRoFormer#8F1: 89.5
natural-language-inference-on-qnli-glue-devRoFormer#16Accuracy: 88.0
machine-translation-on-wmt2014-english-germanRoFormer#42BLEU score: 27.5
sentiment-analysis-on-sst-2-binary-classificationRoFormer#87Dev Accuracy: 90.7