| Benchmark | Model | Rank | Results |
|---|---|---|---|
| language-modelling-on-enwiki8 | SHA-RNN (4 layers, h=1024, attention head per layer) | #26 | Bit per Character (BPC): 1.068Number of params: 54M |
| language-modelling-on-enwiki8 | SHA-RNN (4 layers, h=1024, single attention head) | #27 | Bit per Character (BPC): 1.076Number of params: 52M |
| language-modelling-on-enwiki8 | SHA-LSTM (4 layers, h=1024, no attention head) | #37 | Bit per Character (BPC): 1.33Number of params: 51M |