Revisiting the Shape Convention of Transformer Language Models

Open paper
Benchmark Model Rank Results
question-answering-on-piqaHourglass (1074M)–Accuracy: 68.2
sentence-completion-on-hellaswagHourglass (1074M)–Accuracy: 40.3

Rank counts only results with a code link.