Partially Shuffling the Training Data to Improve Language Models

Benchmark Model Rank Results
language-modelling-on-penn-treebank-wordAWD-LSTM-DOC + Partial Shuffle#14Test perplexity: 52.0Validation perplexity: 53.79Params: 23M
language-modelling-on-penn-treebank-wordAWD-LSTM-MoS + Partial Shuffle#17Test perplexity: 53.92Validation perplexity: 55.89Params: 22M
language-modelling-on-wikitext-2AWD-LSTM-DOC + Partial Shuffle#22Test perplexity: 57.85Validation perplexity: 60.16
language-modelling-on-wikitext-2AWD-LSTM-MoS + Partial Shuffle#24Test perplexity: 59.98Validation perplexity: 62.38