Gated DeltaNet-2: Decoupling Erase and Write in Linear Attention

Open paper
Benchmark Model Rank Results
social-commonsense-reasoning-on-siqaGated DeltaNet-2 (1.3B)#14Accuracy: 41.5
language-modelling-on-lambadaGated DeltaNet-2 (1.3B)#43Accuracy: 50.9Perplexity: 10.43
common-sense-reasoning-on-arc-easyGated DeltaNet-2 (1.3B)#47Accuracy: 72.43
question-answering-on-natural-questionsGated DeltaNet-2 (1.3B)#49EM: 26.31
question-answering-on-boolqGated DeltaNet-2 (1.3B)#84Accuracy: 62.57
question-answering-on-piqaGated DeltaNet-2 (1.3B)#130Accuracy: 72.8
common-sense-reasoning-on-winograndeGated DeltaNet-2 (1.3B)#131Accuracy: 58.56
sentence-completion-on-hellaswagGated DeltaNet-2 (1.3B)#158Accuracy: 58.46

Rank counts only results with a code link.