Memory-efficient Stochastic methods for Memory-based Transformers

Benchmark Model Rank Results
language-modelling-on-enwiki8Skip Cross-Head Transformer-XL#23Bit per Character (BPC): 1.033Number of params: 41M
language-modelling-on-wikitext-103Skip Cross-Head Transformer-XL#48Test perplexity: 22.91Validation perplexity: 21.87