Large Language Models are Zero-Shot Reasoners

Benchmark Model Rank Results
arithmetic-reasoning-on-gsm8kPaLM-540B (few-Shot-cot)#84Accuracy: 58.1Parameters (Billion): 540
arithmetic-reasoning-on-gsm8kFinetuned GPT-3 175B + verifier#90Accuracy: 55.0Parameters (Billion): 175
arithmetic-reasoning-on-gsm8kText-davinci-002-175B (zero-plus-few-Shot-cot (8 samples))#95Accuracy: 51.5Parameters (Billion): 175
arithmetic-reasoning-on-gsm8ktext-davinci-002 175B (2-shot, CoT)#99Accuracy: 41.3Parameters (Billion): 175
arithmetic-reasoning-on-gsm8ktext-davinci-002 175B (0-shot, CoT)#100Accuracy: 40.7Parameters (Billion): 175
arithmetic-reasoning-on-gsm8kPaLM 540B (few-shot)#110Accuracy: 17.9Parameters (Billion): 540
arithmetic-reasoning-on-gsm8kText-davinci-002-175B (0-shot)#114Accuracy: 10.4Parameters (Billion): 175
common-sense-reasoning-on-recordGPT-3 175B (one-shot)#18F1: 90.2
math-word-problem-solving-on-svampPaLM (zero-shot, CoT)#13Execution Accuracy: 62.1
math-word-problem-solving-on-svampPaLM (zero-shot)#14Execution Accuracy: 58.8