| Benchmark | Model | Rank | Results |
|---|---|---|---|
| language-modelling-on-one-billion-word | MDLM (AR baseline) | #1 | PPL: 20.09Number of params: 110M |
| language-modelling-on-one-billion-word | MDLM | #6 | PPL: 23.00Number of params: 110M |
| language-modelling-on-openwebtext | ARM | #1 | eval_perplexity: 17.54parameters: 131M |
| language-modelling-on-openwebtext | MDLM | #8 | eval_perplexity: 22.98parameters: 131M |