MathCoder: Seamless Code Integration in LLMs for Enhanced Mathematical Reasoning

Benchmark Model Rank Results
math-word-problem-solving-on-svampMathCoder-L-70B#5Execution Accuracy: 84.9
math-word-problem-solving-on-mathMathCoder-CL-34B#105Accuracy: 45.2Parameters (Billions): 34
math-word-problem-solving-on-mathMathCoder-L-34B#106Accuracy: 45.1Parameters (Billions): 34
math-word-problem-solving-on-mathMathCoder-CL-13B#126Accuracy: 35.9Parameters (Billions): 13
math-word-problem-solving-on-mathMathCoder-CL-7B#139Accuracy: 30.2Parameters (Billions): 7
math-word-problem-solving-on-mathMathCoder-L-13B#140Accuracy: 29.9Parameters (Billions): 13
math-word-problem-solving-on-mathMathCoder-L-7B#153Accuracy: 23.3Parameters (Billions): 7
arithmetic-reasoning-on-gsm8kMathCoder-L-70B#133Accuracy: 83.9Parameters (Billion): 70
arithmetic-reasoning-on-gsm8kMathCoder-CL-34B#149Accuracy: 81.7Parameters (Billion): 34
arithmetic-reasoning-on-gsm8kMathCoder-CL-13B#185Accuracy: 74.1Parameters (Billion): 7
arithmetic-reasoning-on-gsm8kMathCoder-L-13B#195Accuracy: 72.6Parameters (Billion): 13
arithmetic-reasoning-on-gsm8kMathCoder-CL-7B#204Accuracy: 67.8Parameters (Billion): 7
arithmetic-reasoning-on-gsm8kMathCoder-L-7B#210Accuracy: 64.2Parameters (Billion): 7