Segatron: Segment-Aware Transformer for Language Modeling and Understanding
Open paper
Benchmark
Model
Rank
Results
language-modelling-on-wikitext-103
SegaTransformer-XL
#21
Test perplexity: 17.1
Number of params: 257M
Rank counts only results with a code link.