Exploring the Limits of Language Modeling

Benchmark Model Rank Results
language-modelling-on-one-billion-word10 LSTM+CNN inputs + SNM10-SKIP (ensemble)#10PPL: 23.7Number of params: 43B
language-modelling-on-one-billion-wordLSTM-8192-1024 + CNN Input#17PPL: 30.0Number of params: 1.04B
language-modelling-on-one-billion-wordLSTM-8192-1024#18PPL: 30.6Number of params: 1.8B