Are NLP Models really able to Solve Simple Math Word Problems?

Benchmark Model Rank Results
math-word-problem-solving-on-asdiv-aGraph2Tree with RoBERTa#6Execution Accuracy: 82.2
math-word-problem-solving-on-asdiv-aGTS with RoBERTa#7Execution Accuracy: 81.2
math-word-problem-solving-on-asdiv-aLSTM Seq2Seq with RoBERTa#9Execution Accuracy: 76.9
math-word-problem-solving-on-mawpsGraph2Tree with RoBERTa#10Accuracy (%): 88.7
math-word-problem-solving-on-mawpsGTS with RoBERTa#11Accuracy (%): 88.5
math-word-problem-solving-on-svampGraph2Tree with RoBERTa#20Execution Accuracy: 43.8Accuracy: 43.8
math-word-problem-solving-on-svampGTS with RoBERTa#21Execution Accuracy: 41.0Accuracy: 41.0
math-word-problem-solving-on-svampLSTM Seq2Seq with RoBERTa#22Execution Accuracy: 40.3Accuracy: 40.3
math-word-problem-solving-on-svampTransformer with RoBERTa#24Execution Accuracy: 38.9Accuracy: 38.9