Focus Your Attention (with Adaptive IIR Filters)

Open paper
Benchmark Model Rank Results
language-modelling-on-enwiki8Focus–Bit per Character (BPC): 0.940Number of params: 22M
language-modelling-on-text8Focus–Bit per Character (BPC): 0.98Number of params: 22M

Rank counts only results with a code link.