Fast Parametric Learning with Activation Memorization

Benchmark Model Rank Results
language-modelling-on-wikitext-103LSTMTest perplexity: 36.4Validation perplexity: 36.0
language-modelling-on-wikitext-103LSTM (Hebbian)Test perplexity: 34.3Validation perplexity: 34.1
language-modelling-on-wikitext-103LSTM (Hebbian, Cache)Test perplexity: 29.7Validation perplexity: 29.9
language-modelling-on-wikitext-103LSTM (Hebbian, Cache, MbPA)Test perplexity: 29.2Validation perplexity: 29.0