Multi-Head Low-Rank Attention

Benchmark Model Rank Results
common-sense-reasoning-on-arc-easyMLRA-2 (2.9B)#68Accuracy: 67.89
question-answering-on-boolqMLRA-4 (2.9B)#91Accuracy: 61.74
common-sense-reasoning-on-winograndeMLRA-4 (2.9B)#108Accuracy: 61.48
question-answering-on-piqaMLRA-2 (2.9B)#110Accuracy: 75.52
sentence-completion-on-hellaswagMLRA-4 (2.9B)#142Accuracy: 62.16