ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solving

Benchmark Model Rank Results
math-word-problem-solving-on-mathToRA-Code 34B model (w/ code, SC, k=50)#54Accuracy: 60.0Parameters (Billions): 34
math-word-problem-solving-on-mathToRA 70B (w/ code, SC, k=50)#64Accuracy: 56.9Parameters (Billions): 70
math-word-problem-solving-on-mathToRA-Code 34B (w/ code)#82Accuracy: 50.8Parameters (Billions): 34
math-word-problem-solving-on-mathToRA 70B (w/ code)#87Accuracy: 49.7Parameters (Billions): 70
math-word-problem-solving-on-mathToRA-Code 13B (w/ code)#93Accuracy: 48.1Parameters (Billions): 13
math-word-problem-solving-on-mathToRA-Code 7B (w/ code)#108Accuracy: 44.6Parameters (Billions): 7
math-word-problem-solving-on-mathToRA 13B (w/ code)#116Accuracy: 43.0Parameters (Billions): 13
math-word-problem-solving-on-mathToRA 7B (w/ code)#120Accuracy: 40.1Parameters (Billions): 7
arithmetic-reasoning-on-gsm8kToRA-70B (SC, k=50)#91Accuracy: 88.3Parameters (Billion): 70
arithmetic-reasoning-on-gsm8kToRA-Code-34B (SC, k=50)#115Accuracy: 85.1Parameters (Billion): 34
arithmetic-reasoning-on-gsm8kToRA 70B#128Accuracy: 84.3Parameters (Billion): 70
arithmetic-reasoning-on-gsm8kToRA-Code 34B#156Accuracy: 80.7Parameters (Billion): 34
arithmetic-reasoning-on-gsm8kToRA-Code 13B#181Accuracy: 75.8Parameters (Billion): 13
arithmetic-reasoning-on-gsm8kToRA-Code 7B#194Accuracy: 72.6Parameters (Billion): 7