| Benchmark | Model | Rank | Results |
|---|---|---|---|
| language-modelling-on-enwiki8 | Transformer (24 layers, 8k adaptive span) | #10 | Bit per Character (BPC): 0.98Number of params: 209M |
| language-modelling-on-enwiki8 | Transformer (12 layers, 8k adaptive span) | #19 | Bit per Character (BPC): 1.02Number of params: 39M |
| language-modelling-on-text8 | 24L Transformer + 8K adaptive span | #3 | Bit per Character (BPC): 1.07Number of params: 209M |
| language-modelling-on-text8 | 12L Transformer + 8K adaptive span | #7 | Bit per Character (BPC): 1.11Number of params: 38M |