Towards Looped Models Done Right, Part II: Rethinking at Fixed Points

Benchmark Model Rank Results
multiple-choice-question-answering-mcqa-on-arc-cLoopedLM-P2 learned-depth lambda_H=0.01 (1.6B)#18Accuracy: 40.44
language-modelling-on-lambadaLoopedLM-P2 learned-depth lambda_H=0.01 (1.6B)#37Accuracy: 55.17
question-answering-on-piqaLoopedLM-P2 learned-depth lambda_H=0.01 (1.6B)#115Accuracy: 73.72