Scaling Inherently Interpretable Language Models

Benchmark Model Rank Results
common-sense-reasoning-on-winograndeSteerling-8B#96Accuracy: 64.2
question-answering-on-piqaSteerling-8B#107Accuracy: 75.9
sentence-completion-on-hellaswagSteerling-8B#109Accuracy: 70.3
code-generation-on-humanevalSteerling-8B#139Pass@1: 8.5
code-generation-on-mbppSteerling-8B#163Accuracy: 11
math-word-problem-solving-on-mathSteerling-8B#183Accuracy: 8.0
arithmetic-reasoning-on-gsm8kSteerling-8B#246Accuracy: 44.4