Disentangling Representation Evolution in Transformers through Directional Decomposition

Benchmark Model Rank Results
zero-shot-question-answering-on-openbookqaTransformer Geometry 2.7B V-Para (zero-shot)#18Accuracy: 38.2
common-sense-reasoning-on-arc-easyTransformer Geometry 2.7B V-Para (zero-shot)#77Accuracy: 60.4
question-answering-on-piqaTransformer Geometry 2.7B V-Para (zero-shot)#78Accuracy: 77.6
question-answering-on-boolqTransformer Geometry 2.7B V-Para (zero-shot)#79Accuracy: 64.4
common-sense-reasoning-on-winograndeTransformer Geometry 2.7B V-Para (5-shot)#104Accuracy: 62.6
sentence-completion-on-hellaswagTransformer Geometry 2.7B V-Para (10-shot)#124Accuracy: 67.1