| Benchmark | Model | Rank | Results |
|---|---|---|---|
| language-modelling-on-enwiki8 | Compressive Transformer (24 layers) | #7 | Bit per Character (BPC): 0.97Number of params: 277M |
| language-modelling-on-hutter-prize | Compressive Transformer | #2 | Bit per Character (BPC): 0.97 |
| language-modelling-on-wikitext-103 | Compressive Transformer (18L, M=1024) | #20 | Test perplexity: 17.1Validation perplexity: 16.0 |