| auto-debugging-on-big-bench-lite | PaLM 62B (few-shot, k=5) | #1 | Exact string match: 38.2 |
| auto-debugging-on-big-bench-lite | PaLM 540B (few-shot, k=5) | #2 | Exact string match: 38.2 |
| auto-debugging-on-big-bench-lite | PaLM 8B (few-shot, k=5) | #3 | Exact string match: 14.7 |
| common-sense-reasoning-on-big-bench-known-unknowns | PaLM-540B (few-shot, k=5) | #1 | Accuracy: 73.9 |
| common-sense-reasoning-on-big-bench-winowhy | PaLM-540B (few-shot, k=5) | #1 | Accuracy: 65.9 |
| common-sense-reasoning-on-big-bench-winowhy | PaLM-62B (few-shot, k=5) | #3 | Accuracy: 61.0 |
| coreference-resolution-on-winograd-schema-challenge | PaLM 540B (fine-tuned) | #1 | Accuracy: 100 |
| coreference-resolution-on-winograd-schema-challenge | PaLM 540B (5-shot) | #9 | Accuracy: 89.5 |
| coreference-resolution-on-winograd-schema-challenge | PaLM 540B (0-shot) | #10 | Accuracy: 89.1 |
| coreference-resolution-on-winograd-schema-challenge | PaLM 540B (1-shot) | #14 | Accuracy: 86.3 |
| extreme-summarization-on-gem-xsum | PaLM (finetuning)-540B | #1 | ROUGE-2: 21.2Parameters: 540 B |
| extreme-summarization-on-gem-xsum | T5-XXL | #2 | ROUGE-2: 21.0 |
| extreme-summarization-on-gem-xsum | PaLM (finetuning)-62B | #3 | ROUGE-2: 18.5Parameters: 62 B |
| language-modelling-on-lambada | PaLM-540B (Few-Shot) | #1 | Accuracy: 89.7 |
| language-modelling-on-lambada | PaLM-540B (One-Shot) | #6 | Accuracy: 81.8 |
| language-modelling-on-lambada | PaLM-540B (Zero-Shot) | #11 | Accuracy: 77.9 |
| logical-reasoning-on-big-bench-strategyqa | PaLM-540B (few-shot, k=5) | #1 | Accuracy: 73.9 |
| logical-reasoning-on-big-bench-strategyqa | PaLM-62B (few-shot, k=5) | #3 | Accuracy: 65.4 |
| memorization-on-big-bench-hindu-knowledge | PaLM-540B (few-shot, k=5) | #1 | Accuracy: 95.4 |
| memorization-on-big-bench-hindu-knowledge | PaLM-62B (few-shot, k=5) | #3 | Accuracy: 77.7 |
| multiple-choice-question-answering-mcqa-on-arc-e | PaLM 540B | #1 | Accuracy: 88.4 |
| multiple-choice-question-answering-mcqa-on-arc-e | PaLM 62B | #2 | Accuracy: 80.6 |
| multiple-choice-question-answering-mcqa-on-arc-e | PaLM 8B | #9 | Accuracy: 71.3 |
| multiple-choice-question-answering-mcqa-on-big-bench-novel-concepts | PaLM-540B (few-shot, k=5) | #1 | Accuracy: 71.9 |
| multiple-choice-question-answering-mcqa-on-big-bench-novel-concepts | PaLM-62B (few-shot, k=5) | #3 | Accuracy: 59.4 |
| natural-language-inference-on-commitmentbank | PaLM 540B (finetuned) | #1 | Accuracy: 100F1: 100 |
| natural-language-inference-on-rte | PaLM 540B (fine-tuned) | #1 | Accuracy: 95.7% |
| natural-language-inference-on-rte | PaLM 540B (5-shot) | #35 | Accuracy: 79.6% |
| natural-language-inference-on-rte | PaLM 540B (1-shot) | #38 | Accuracy: 78.7% |
| natural-language-inference-on-rte | PaLM 540B (0-shot) | #45 | Accuracy: 72.9% |
| question-answering-on-copa | PaLM 540B (finetuned) | #1 | Accuracy: 100 |
| question-answering-on-multirc | PaLM 540B (finetuned) | #1 | F1: 90.1EM: 69.2 |
| multiple-choice-question-answering-mcqa-on-arc-c | PaLM 540B | #2 | Accuracy: 65.9 |
| multiple-choice-question-answering-mcqa-on-arc-c | PaLM 62B | #4 | Accuracy: 53.7 |
| multiple-choice-question-answering-mcqa-on-arc-c | PaLM 8B | #19 | Accuracy: 40.2 |
| word-sense-disambiguation-on-words-in-context | PaLM 540B (finetuned) | #2 | Accuracy: 78.8 |
| common-sense-reasoning-on-record | PaLM 540B (finetuned) | #3 | EM: 94.0F1: 94.6 |
| question-answering-on-boolq | PaLM 540B (fine-tuned) | #3 | Accuracy: 92.2 |
| cross-lingual-question-answering-on-tydiqa-goldp | PaLM-540B (CoT) | #5 | EM: 52.9 |
| question-answering-on-triviaqa | PaLM-540B (Few-Shot) | #7 | EM: 81.4 |
| question-answering-on-triviaqa | PaLM-540B (One-Shot) | #8 | EM: 81.4 |
| question-answering-on-triviaqa | PaLM-540B (Zero-Shot) | #11 | EM: 76.9 |
| question-answering-on-obqa | PaLM 540B (zero-shot) | #8 | Accuracy: 53.4 |
| question-answering-on-obqa | PaLM 62B (zero-shot) | #9 | Accuracy: 50.4 |
| question-answering-on-webquestions | PaLM-540B (Few-Shot) | #8 | EM: 43.5 |
| question-answering-on-webquestions | PaLM-540B (One-Shot) | #27 | EM: 22.6 |
| question-answering-on-webquestions | PaLM-540B (Zero-Shot) | #30 | EM: 10.6 |
| multi-task-language-understanding-on-mgsm | PaLM 540B | #12 | Average (%): 55.0 |
| reading-comprehension-on-race | PaLM 540B (zero-shot) | #13 | Accuracy (Middle): 68.1Accuracy (High): 49.1 |
| reading-comprehension-on-race | PaLM 62B (zero-shot) | #15 | Accuracy (Middle): 64.3Accuracy (High): 47.5 |
| reading-comprehension-on-race | PaLM 8B (zero-shot) | #20 | Accuracy (Middle): 57.9Accuracy (High): 42.3 |
| question-answering-on-piqa | PaLM 540B | #15 | Accuracy: 85.2 |
| question-answering-on-piqa | PaLM 62B | #34 | Accuracy: 82.3 |
| question-answering-on-piqa | PaLM 8B | #84 | Accuracy: 77.5 |
| common-sense-reasoning-on-winogrande | PaLM 540B (0-shot) | #26 | Accuracy: 81.1 |
| common-sense-reasoning-on-winogrande | PaLM 62B (0-shot) | #47 | Accuracy: 77.0 |
| common-sense-reasoning-on-winogrande | PaLM-cont 62B (0-shot) | #48 | Accuracy: 77.0 |
| multi-task-language-understanding-on-mmlu | PaLM 540B | #29 | Average (%): 69.3 |
| multi-task-language-understanding-on-mmlu | PaLM 62B | #59 | Average (%): 53.7 |
| multi-task-language-understanding-on-mmlu | PaLM 8B | #87 | Average (%): 25.3 |
| question-answering-on-natural-questions | PaLM-540B (Few-Shot, k=64) | #32 | EM: 39.6 |
| question-answering-on-natural-questions | PaLM-540B (One-Shot) | #45 | EM: 29.3 |
| question-answering-on-natural-questions | PaLM-540B (Zero-Shot) | #59 | EM: 21.2 |
| sentence-completion-on-hellaswag | PaLM-540B (Few-Shot) | #43 | Accuracy: 83.8 |
| sentence-completion-on-hellaswag | PaLM-540B (1-shot) | #46 | Accuracy: 83.6 |
| sentence-completion-on-hellaswag | PaLM-540B (0-shot) | #49 | Accuracy: 83.4 |
| code-generation-on-mbpp | PaLM Coder 540B | #118 | Accuracy: 47 |
| code-generation-on-mbpp | PaLM 540B | #141 | Accuracy: 36.8 |
| code-generation-on-humaneval | PaLM 540B | #128 | Pass@1: 26.2 |