| language-modelling-on-enwiki8 | Transformer-XL (24 layers, RMS dynamic eval, decay) | #2 | Bit per Character (BPC): 0.940Number of params: 277M |
| language-modelling-on-hutter-prize | Transformer-XL + RMS dynamic eval | #1 | Bit per Character (BPC): 0.94Number of params: 277M |
| language-modelling-on-text8 | Transformer-XL + RMS dynamic eval + decay | #2 | Bit per Character (BPC): 1.038Number of params: 277M |
| language-modelling-on-wikitext-103 | Transformer-XL (RMS dynamic eval) | #13 | Test perplexity: 16.4Validation perplexity: 15.8… |
| language-modelling-on-wikitext-103 | Transformer-XL (SGD dynamic eval) | #19 | Test perplexity: 17.0Validation perplexity: 16.3… |