Transcending Scaling Laws with 0.1% Extra Compute

Benchmark Model Rank Results
arithmetic-reasoning-on-gsm8kU-PaLMAccuracy: 58.5Parameters (Billion): 540
cross-lingual-question-answering-on-tydiqaU-PaLM 62B (fine-tuned)EM: 78.4F1: 88.5
cross-lingual-question-answering-on-tydiqaU-PaLM-540B (CoT)EM: 54.6
multi-task-language-understanding-on-mgsmU-PaLM 540B (CoT)Average (%): 49.9
question-answering-on-strategyqaMinerva 540BAccuracy: 61.9
question-answering-on-strategyqaPaLM 540BAccuracy: 76.4
question-answering-on-strategyqaU-PaLM 540BAccuracy: 76.6