Measuring Mathematical Problem Solving With the MATH Dataset

Benchmark Model Rank Results
math-word-problem-solving-on-mathGPT-2 (1.5B)#185Accuracy: 6.9Parameters (Billions): 1.5
math-word-problem-solving-on-mathGPT-2 (0.7B)#187Accuracy: 6.4Parameters (Billions): 0.7
math-word-problem-solving-on-mathGPT-2 (0.3B)#188Accuracy: 6.2Parameters (Billions): 0.3
math-word-problem-solving-on-mathGPT-3 13B#189Accuracy: 5.6Parameters (Billions): 13
math-word-problem-solving-on-mathGPT-2 (0.1B)#191Accuracy: 5.4Parameters (Billions): 0.1
math-word-problem-solving-on-mathGPT-3-175B (few-shot)#192Accuracy: 5.2Parameters (Billions): 175
math-word-problem-solving-on-mathGPT-3-13B (few-shot)#196Accuracy: 3.0Parameters (Billions): 13
math-word-problem-solving-on-mathGPT-3 2.7B#197Accuracy: 2.9Parameters (Billions): 2.7