LoopRPT: Reinforcement Pre-Training for Looped Language Models

Benchmark Model Rank Results
arithmetic-reasoning-on-gsm8kOuro (2.6B) LoopRPT–Accuracy: 85.36
code-generation-on-humanevalOuro (2.6B) LoopRPT–Pass@1: 76.83
code-generation-on-mbppOuro (2.6B) LoopRPT–Accuracy: 77.24
common-sense-reasoning-on-winograndeOuro (2.6B) LoopRPT–Accuracy: 76.47
multi-task-language-understanding-on-mmlu-proOuro (2.6B) LoopRPT–Accuracy: 54.19
sentence-completion-on-hellaswagOuro (2.6B) LoopRPT–Accuracy: 80.03