Not All Memories are Created Equal: Learning to Forget by Expiring
Open paper
Benchmark
Model
Rank
Results
language-modelling-on-enwiki8
Expire-Span (24 layers)
#4
Bit per Character (BPC): 0.95
Number of params: 208M
Rank counts only results with a code link.