Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations

Open paper
Benchmark Model Rank Results
arithmetic-reasoning-on-gsm8kShepherd+Mistral-7B (SFT on MetaMATH + PRM RL+ PRM rerank, k=256)#84Accuracy: 89.1Parameters (Billion): 7
arithmetic-reasoning-on-gsm8kShepherd + Mistral-7B (SFT on MetaMATH + PRM RL)#131Accuracy: 84.1Parameters (Billion): 7
math-word-problem-solving-on-mathShepherd + DeepSeek-67B (SFT on MetaMATH + PRM rerank, k=256)#94Accuracy: 48.1Parameters (Billions): 67
math-word-problem-solving-on-mathShepherd+Mistral-7B (SFT on MetaMATH + PRM RL+ PRM rerank, k=256)#113Accuracy: 43.5Parameters (Billions): 7
math-word-problem-solving-on-mathShepherd + Mistral-7B (SFT on MetaMATH + PRM RL)#132Accuracy: 33.0Parameters (Billions): 7

Rank counts only results with a code link.