Large Language Models are Zero-Shot Reasoners

Benchmark Model Rank Results
math-word-problem-solving-on-svampPaLM (zero-shot, CoT)#13Execution Accuracy: 62.1
math-word-problem-solving-on-svampPaLM (zero-shot)#14Execution Accuracy: 58.8
common-sense-reasoning-on-recordGPT-3 175B (one-shot)#18F1: 90.2
arithmetic-reasoning-on-gsm8kPaLM-540B (few-Shot-cot)#218Accuracy: 58.1Parameters (Billion): 540
arithmetic-reasoning-on-gsm8kFinetuned GPT-3 175B + verifier#226Accuracy: 55.0Parameters (Billion): 175
arithmetic-reasoning-on-gsm8kText-davinci-002-175B (zero-plus-few-Shot-cot (8 samples))#233Accuracy: 51.5Parameters (Billion): 175
arithmetic-reasoning-on-gsm8ktext-davinci-002 175B (2-shot, CoT)#248Accuracy: 41.3Parameters (Billion): 175
arithmetic-reasoning-on-gsm8ktext-davinci-002 175B (0-shot, CoT)#249Accuracy: 40.7Parameters (Billion): 175
arithmetic-reasoning-on-gsm8kPaLM 540B (few-shot)#267Accuracy: 17.9Parameters (Billion): 540
arithmetic-reasoning-on-gsm8kText-davinci-002-175B (0-shot)#272Accuracy: 10.4Parameters (Billion): 175