PaLM: Scaling Language Modeling with Pathways

Benchmark Model Rank Results
auto-debugging-on-big-bench-litePaLM 62B (few-shot, k=5)#1Exact string match: 38.2
auto-debugging-on-big-bench-litePaLM 540B (few-shot, k=5)#2Exact string match: 38.2
auto-debugging-on-big-bench-litePaLM 8B (few-shot, k=5)#3Exact string match: 14.7
code-generation-on-mbppPaLM Coder 540B#60Accuracy: 47
code-generation-on-mbppPaLM 540B#75Accuracy: 36.8
common-sense-reasoning-on-big-bench-knownPaLM-540B (few-shot, k=5)#1Accuracy: 73.9
common-sense-reasoning-on-big-bench-winowhyPaLM-540B (few-shot, k=5)#1Accuracy: 65.9
common-sense-reasoning-on-big-bench-winowhyPaLM-62B (few-shot, k=5)#3Accuracy: 61.0
common-sense-reasoning-on-recordPaLM 540B (finetuned)#3EM: 94.0F1: 94.6
common-sense-reasoning-on-winograndePaLM 540B (0-shot)#14Accuracy: 81.1
common-sense-reasoning-on-winograndePaLM 62B (0-shot)#20Accuracy: 77.0
common-sense-reasoning-on-winograndePaLM-cont 62B (0-shot)#21Accuracy: 77.0
coreference-resolution-on-winograd-schemaPaLM 540B (fine-tuned)#1Accuracy: 100
coreference-resolution-on-winograd-schemaPaLM 540B (5-shot)#9Accuracy: 89.5
coreference-resolution-on-winograd-schemaPaLM 540B (0-shot)#10Accuracy: 89.1
coreference-resolution-on-winograd-schemaPaLM 540B (1-shot)#14Accuracy: 86.3
cross-lingual-question-answering-on-tydiqaPaLM-540B (CoT)#5EM: 52.9
extreme-summarization-on-gem-xsumPaLM (finetuning)-540B#1ROUGE-2: 21.2Parameters: 540 B
extreme-summarization-on-gem-xsumT5-XXL#2ROUGE-2: 21.0
extreme-summarization-on-gem-xsumPaLM (finetuning)-62B#3ROUGE-2: 18.5Parameters: 62 B
language-modelling-on-lambadaPaLM-540B (Few-Shot)#1Accuracy: 89.7
language-modelling-on-lambadaPaLM-540B (One-Shot)#5Accuracy: 81.8
language-modelling-on-lambadaPaLM-540B (Zero-Shot)#10Accuracy: 77.9
logical-reasoning-on-big-bench-strategyqaPaLM-540B (few-shot, k=5)#1Accuracy: 73.9
logical-reasoning-on-big-bench-strategyqaPaLM-62B (few-shot, k=5)#3Accuracy: 65.4
memorization-on-big-bench-hindu-knowledgePaLM-540B (few-shot, k=5)#1Accuracy: 95.4
memorization-on-big-bench-hindu-knowledgePaLM-62B (few-shot, k=5)#3Accuracy: 77.7
multi-task-language-understanding-on-mgsmPaLM 540B#6Average (%): 55.0
multiple-choice-question-answering-mcqa-on-31PaLM-540B (few-shot, k=5)#1Accuracy: 71.9
multiple-choice-question-answering-mcqa-on-31PaLM-62B (few-shot, k=5)#3Accuracy: 59.4
natural-language-inference-on-commitmentbankPaLM 540B (finetuned)#1Accuracy: 100F1: 100
natural-language-inference-on-rtePaLM 540B (fine-tuned)#1Accuracy: 95.7%
natural-language-inference-on-rtePaLM 540B (5-shot)#32Accuracy: 79.6%
natural-language-inference-on-rtePaLM 540B (1-shot)#35Accuracy: 78.7%
natural-language-inference-on-rtePaLM 540B (0-shot)#42Accuracy: 72.9%
question-answering-on-boolqPaLM 540B (fine-tuned)#3Accuracy: 92.2
question-answering-on-copaPaLM 540B (finetuned)#1Accuracy: 100
question-answering-on-multircPaLM 540B (finetuned)#1F1: 90.1EM: 69.2
question-answering-on-natural-questionsPaLM-540B (Few-Shot, k=64)#19EM: 39.6
question-answering-on-natural-questionsPaLM-540B (One-Shot)#28EM: 29.3
question-answering-on-natural-questionsPaLM-540B (Zero-Shot)#33EM: 21.2
question-answering-on-obqaPaLM 540B (zero-shot)#8Accuracy: 53.4
question-answering-on-obqaPaLM 62B (zero-shot)#9Accuracy: 50.4
question-answering-on-triviaqaPaLM-540B (Few-Shot)#5EM: 81.4
question-answering-on-triviaqaPaLM-540B (One-Shot)#6EM: 81.4
question-answering-on-triviaqaPaLM-540B (Zero-Shot)#9EM: 76.9
question-answering-on-webquestionsPaLM-540B (Few-Shot)#8EM: 43.5
question-answering-on-webquestionsPaLM-540B (One-Shot)#27EM: 22.6
question-answering-on-webquestionsPaLM-540B (Zero-Shot)#30EM: 10.6
reading-comprehension-on-racePaLM 540B (zero-shot)#8Accuracy (Middle): 68.1Accuracy (High): 49.1
reading-comprehension-on-racePaLM 62B (zero-shot)#10Accuracy (Middle): 64.3Accuracy (High): 47.5
reading-comprehension-on-racePaLM 8B (zero-shot)#15Accuracy (Middle): 57.9Accuracy (High): 42.3
sentence-completion-on-hellaswagPaLM-540B (Few-Shot)#22Accuracy: 83.8
sentence-completion-on-hellaswagPaLM-540B (1-shot)#23Accuracy: 83.6
sentence-completion-on-hellaswagPaLM-540B (0-shot)#25Accuracy: 83.4
word-sense-disambiguation-on-words-in-contextPaLM 540B (finetuned)#2Accuracy: 78.8