| Benchmark | Model | Rank | Results |
|---|---|---|---|
| language-modelling-on-one-billion-word | High-Budget MoE | #15 | PPL: 28.0Number of params: 5B |
| language-modelling-on-one-billion-word | Low-Budget MoE | #20 | PPL: 34.1Number of params: 5B |
| machine-translation-on-wmt2014-english-french | MoE | #30 | BLEU score: 40.56Hardware Burden: 142G |
| machine-translation-on-wmt2014-english-german | MoE | #56 | BLEU score: 26.03Hardware Burden: 24G |