Parallax: Parameterized Local Linear Attention for Language Modeling

Benchmark Model Rank Results
zero-shot-question-answering-on-sciqParallax (1.7B) (LM Evaluation Harness)#5Accuracy: 88.7
common-sense-reasoning-on-arc-easyParallax (1.7B)#43Accuracy: 73.27
language-modelling-on-lambadaParallax (1.7B)#45Accuracy: 50.26Perplexity: 10.8
question-answering-on-boolqParallax (1.7B)#78Accuracy: 64.59
question-answering-on-piqaParallax (1.7B)#101Accuracy: 76.39
common-sense-reasoning-on-winograndeParallax (1.7B)#112Accuracy: 60.77
sentence-completion-on-hellaswagParallax (1.7B)#133Accuracy: 64.54