Do Language Models Need a Trainable Input Embedding Table? Fixed Minimal Token Codes at 1.7B-Class Scale
Open paper
Benchmark
Model
Rank
Results
language-modelling-on-lambada
ab_ext_binary16
–
Accuracy: 42.75
Rank counts only results with a code link.