| language-modelling-on-wikitext-103 | [?]-former (SM) | #14 | Test perplexity: 16.61 |
| language-modelling-on-wikitext-103 | -former (SM) | #15 | Test perplexity: 16.61 |
| language-modelling-on-wikitext-103 | ∞-former (Sticky memories + initialized GPT-2 Small) | #16 | Test perplexity: 16.61 |
| language-modelling-on-wikitext-103 | ∞-former (initialized GPT-2 Small) | #17 | Test perplexity: 16.64 |
| language-modelling-on-wikitext-103 | [?]-former (Sticky memories) | #54 | Test perplexity: 24.22 |
| language-modelling-on-wikitext-103 | \infty-former (Sticky memories) | #55 | Test perplexity: 24.22 |
| language-modelling-on-wikitext-103 | ∞-former (Sticky memories) | #56 | Test perplexity: 24.22 |