| language-modelling-on-enwiki8 | Transformer-XL (24 layers) | #11 | Bit per Character (BPC): 0.99Number of params: 277M |
| language-modelling-on-enwiki8 | Transformer-XL (18 layers) | #22 | Bit per Character (BPC): 1.03Number of params: 88M |
| language-modelling-on-enwiki8 | Transformer-XL (12 layers) | #25 | Bit per Character (BPC): 1.06Number of params: 41M |
| language-modelling-on-hutter-prize | 24-layer Transformer-XL | #4 | Bit per Character (BPC): 0.99Number of params: 277M |
| language-modelling-on-hutter-prize | 18-layer Transformer-XL | #7 | Bit per Character (BPC): 1.03Number of params: 88M |
| language-modelling-on-hutter-prize | 12-layer Transformer-XL | #9 | Bit per Character (BPC): 1.06Number of params: 41M |
| language-modelling-on-one-billion-word | Transformer-XL Large | #4 | PPL: 21.8Number of params: 0.8B |
| language-modelling-on-one-billion-word | Transformer-XL Base | #8 | PPL: 23.5Number of params: 0.46B |
| language-modelling-on-penn-treebank-word | Transformer-XL | #21 | Test perplexity: 54.55Validation perplexity: 56.72Params: 24M |
| language-modelling-on-text8 | Transformer-XL - 24 layers | #4 | Bit per Character (BPC): 1.08Number of params: 277M |
| language-modelling-on-wikitext-103 | Transformer-XL Large | #33 | Test perplexity: 18.3Validation perplexity: 18.2… |
| language-modelling-on-wikitext-103 | Transformer-XL Standard | #52 | Test perplexity: 24.0Validation perplexity: 23.1… |