| language-modelling-on-enwiki8 | All-attention network (18 layers) | #18 | Bit per Character (BPC): 1.01Number of params: 39M |
| language-modelling-on-enwiki8 | All-attention network (36 layers) | #40 | Number of params: 114M |
| language-modelling-on-text8 | All-attention network - 36 layers | #5 | Bit per Character (BPC): 1.08Number of params: 114M |
| language-modelling-on-text8 | All-attention network - 18 layers | #8 | Bit per Character (BPC): 1.11Number of params: 38M |
| language-modelling-on-wikitext-103 | All-attention network (36 layers) | #43 | Test perplexity: 20.6Validation perplexity: 19.7… |