Mid-Training with Self-Generated Data Improves Reinforcement Learning in Language Models

Benchmark Model Rank Results
code-generation-on-humanevalLlama 3.2–3B–Instruct (mid-trained, n=64, RL)–Pass@1: 52.82
mathematical-reasoning-on-aime-2025Llama 3.2–3B–Instruct (mid-trained, n=64)–Accuracy: 0.44
mathematical-reasoning-on-aime24Llama 3.2–3B–Instruct (mid-trained, n=32)–Acc: 2.64
mathematical-reasoning-on-math-500Llama 3.2–3B–Instruct (mid-trained, n=64)–Accuracy: 38.11
mathematical-reasoning-on-olympiadbenchLlama 3.2–3B–Instruct (mid-trained, n=64)–Accuracy: 8.66