| Benchmark | Model | Rank | Results |
|---|---|---|---|
| language-modelling-on-the-pile | Larger Transformer 771M (fine-tuned) | – | Test perplexity: 10 |
| language-modelling-on-the-pile | Larger Transformer 771M (pre-trained) | – | Test perplexity: 28.1 |
| language-modelling-on-the-pile | Smaller Transformer 126M (fine-tuned) | – | Test perplexity: 12 |
| language-modelling-on-the-pile | Smaller Transformer 126M (pre-trained) | – | Test perplexity: 33 |