H-Transformer-1D: Fast One-Dimensional Hierarchical Attention for Sequences

Benchmark Model Rank Results
language-modelling-on-one-billion-wordH-Transformer-1D Nr=16 (Base)#24Number of params: 53MValidation perplexity: 23.95
language-modelling-on-one-billion-wordH-Transformer-1D Nr=16 (Large)#25Number of params: 144MValidation perplexity: 20.25