Multiplicative LSTM for sequence modelling

Benchmark Model Rank Results
language-modelling-on-enwiki8Large mLSTM#32Bit per Character (BPC): 1.24Number of params: 46M
language-modelling-on-hutter-prizeLarge mLSTM +emb +WN +VD#14Bit per Character (BPC): 1.24Number of params: 46M
language-modelling-on-text8Large mLSTM +emb +WN +VD#17Bit per Character (BPC): 1.27Number of params: 45M
language-modelling-on-text8Unregularised mLSTM#20Bit per Character (BPC): 1.40Number of params: 45M