| language-modelling-on-wikitext-103 | [?]-former (SM) | #18 | Test perplexity: 16.61 |
| language-modelling-on-wikitext-103 | -former (SM) | #19 | Test perplexity: 16.61 |
| language-modelling-on-wikitext-103 | ∞-former (Sticky memories + initialized GPT-2 Small) | #20 | Test perplexity: 16.61 |
| language-modelling-on-wikitext-103 | ∞-former (initialized GPT-2 Small) | #21 | Test perplexity: 16.64 |
| language-modelling-on-wikitext-103 | [?]-former (Sticky memories) | #58 | Test perplexity: 24.22 |
| language-modelling-on-wikitext-103 | \infty-former (Sticky memories) | #59 | Test perplexity: 24.22 |
| language-modelling-on-wikitext-103 | ∞-former (Sticky memories) | #60 | Test perplexity: 24.22 |