MathCoder: Seamless Code Integration in LLMs for Enhanced Mathematical Reasoning

Benchmark Model Rank Results
arithmetic-reasoning-on-gsm8kMathCoder-L-70B#37Accuracy: 83.9Parameters (Billion): 70
arithmetic-reasoning-on-gsm8kMathCoder-CL-34B#44Accuracy: 81.7Parameters (Billion): 34
arithmetic-reasoning-on-gsm8kMathCoder-CL-13B#63Accuracy: 74.1Parameters (Billion): 7
arithmetic-reasoning-on-gsm8kMathCoder-L-13B#70Accuracy: 72.6Parameters (Billion): 13
arithmetic-reasoning-on-gsm8kMathCoder-CL-7B#76Accuracy: 67.8Parameters (Billion): 7
arithmetic-reasoning-on-gsm8kMathCoder-L-7B#80Accuracy: 64.2Parameters (Billion): 7
math-word-problem-solving-on-mathMathCoder-CL-34B#52Accuracy: 45.2Parameters (Billions): 34
math-word-problem-solving-on-mathMathCoder-L-34B#53Accuracy: 45.1Parameters (Billions): 34
math-word-problem-solving-on-mathMathCoder-CL-13B#65Accuracy: 35.9Parameters (Billions): 13
math-word-problem-solving-on-mathMathCoder-CL-7B#74Accuracy: 30.2Parameters (Billions): 7
math-word-problem-solving-on-mathMathCoder-L-13B#75Accuracy: 29.9Parameters (Billions): 13
math-word-problem-solving-on-mathMathCoder-L-7B#84Accuracy: 23.3Parameters (Billions): 7
math-word-problem-solving-on-svampMathCoder-L-70B#5Execution Accuracy: 84.9