OpenMathInstruct-1: A 1.8 Million Math Instruction Tuning Dataset

Open paper
Benchmark Model Rank Results
arithmetic-reasoning-on-gsm8kOpenMath-CodeLlama-70B (w/ code, SC, k=50)#13Accuracy: 90.8Parameters (Billion): 70
arithmetic-reasoning-on-gsm8kOpenMath-Llama2-70B (w/ code, SC, k=50)#15Accuracy: 90.1Parameters (Billion): 70
arithmetic-reasoning-on-gsm8kOpenMath-CodeLlama-34B (w/ code, SC, k=50)#22Accuracy: 88.0Parameters (Billion): 34
arithmetic-reasoning-on-gsm8kOpenMath-Mistral-7B (w/ code, SC, k=50)#25Accuracy: 86.9Parameters (Billion): 7
arithmetic-reasoning-on-gsm8kOpenMath-CodeLlama-13B (w/ code, SC, k=50)#26Accuracy: 86.8Parameters (Billion): 13
arithmetic-reasoning-on-gsm8kOpenMath-CodeLlama-7B (w/ code, SC, k=50)#30Accuracy: 84.8Parameters (Billion): 7
arithmetic-reasoning-on-gsm8kOpenMath-Llama2-70B (w/ code)#32Accuracy: 84.7Parameters (Billion): 70
arithmetic-reasoning-on-gsm8kOpenMath-CodeLlama-70B (w/ code)#33Accuracy: 84.6Parameters (Billion): 70
arithmetic-reasoning-on-gsm8kOpenMath-CodeLlama-34B (w/ code)#50Accuracy: 80.7Parameters (Billion): 34
arithmetic-reasoning-on-gsm8kOpenMath-Mistral-7B (w/ code)#53Accuracy: 80.2Parameters (Billion): 7
arithmetic-reasoning-on-gsm8kOpenMath-CodeLlama-13B (w/ code)#54Accuracy: 78.8Parameters (Billion): 13
arithmetic-reasoning-on-gsm8kOpenMath-CodeLlama-7B (w/ code)#60Accuracy: 75.9Parameters (Billion): 7
math-word-problem-solving-on-asdiv-aOpenMath-CodeLlama-70B (w/ code)#5Execution Accuracy: 84.7
math-word-problem-solving-on-mathOpenMath-CodeLlama-70B (w/ code, SC, k=50)#17Accuracy: 60.4Parameters (Billions): 70
math-word-problem-solving-on-mathOpenMath-CodeLlama-34B (w/ code, SC, k=50)#18Accuracy: 60.2Parameters (Billions): 34
math-word-problem-solving-on-mathOpenMath-Llama2-70B (w/ code, SC, k=50)#21Accuracy: 58.3Parameters (Billions): 70
math-word-problem-solving-on-mathOpenMath-CodeLlama-13B (w/ code, SC, k=50)#23Accuracy: 57.6Parameters (Billions): 13
math-word-problem-solving-on-mathOpenMath-Mistral-7B (w/ code, SC, k=50)#24Accuracy: 57.2Parameters (Billions): 7
math-word-problem-solving-on-mathOpenMath-CodeLlama-7B (w/ code, SC, k=50)#27Accuracy: 55.6Parameters (Billions): 7
math-word-problem-solving-on-mathOpenMath-CodeLlama-70B (w/ code)#38Accuracy: 50.7Parameters (Billions): 70
math-word-problem-solving-on-mathOpenMath-CodeLlama-34B (w/ code)#43Accuracy: 48.3Parameters (Billions): 34
math-word-problem-solving-on-mathOpenMath-Llama2-70B (w/ code)#48Accuracy: 46.3Parameters (Billions): 70
math-word-problem-solving-on-mathOpenMath-CodeLlama-13B (w/ code)#49Accuracy: 45.5Parameters (Billions): 13
math-word-problem-solving-on-mathOpenMath-Mistral-7B (w/ code)#56Accuracy: 44.5Parameters (Billions): 7
math-word-problem-solving-on-mathOpenMath-CodeLlama-7B (w/ code)#58Accuracy: 43.6Parameters (Billions): 7
math-word-problem-solving-on-mawpsOpenMath-CodeLlama-70B (w/ code)#1Accuracy (%): 95.7
math-word-problem-solving-on-svampOpenMath-CodeLlama-70B (w/ code)#4Execution Accuracy: 87.8

Rank counts only results with a code link.