Zamba: A Compact 7B SSM Hybrid Model

Benchmark Model Rank Results
zero-shot-question-answering-on-openbookqaZamba 7B#14Accuracy: 44.6
question-answering-on-boolqZamba 7B#33Accuracy: 83.6
common-sense-reasoning-on-arc-easyZamba 7B#37Accuracy: 74.5
question-answering-on-piqaZamba 7B#43Accuracy: 81.37
common-sense-reasoning-on-winograndeZamba 7B#51Accuracy: 76.4
multi-task-language-understanding-on-mmluZamba 7B (5-shot)#55Average (%): 57.72
sentence-completion-on-hellaswagZamba 7B#73Accuracy: 80.24