Mixture-of-Depths Attention

Open paper
Benchmark Model Rank Results
common-sense-reasoning-on-arc-easyMoDA (1.5B)#46Accuracy: 72.81
question-answering-on-piqaMoDA (1.5B)#96Accuracy: 76.82
sentence-completion-on-hellaswagMoDA (1.5B)#129Accuracy: 66.24

Rank counts only results with a code link.