Measuring and Narrowing the Compositionality Gap in Language Models

Benchmark Model Rank Results
question-answering-on-feverSelf-Ask#2EM: 64.2
question-answering-on-bamboogleSelf-ask (GPT-3; davinci-002) + Google Search#3Accuracy: 60.0
question-answering-on-bamboogleSelf-ask (GPT-3; davinci-002)#4Accuracy: 57.6
question-answering-on-bamboogleChain-of-Thought (GPT-3; davinci-002)#5Accuracy: 46.4
question-answering-on-bamboogleDirect Prompting (GPT-3; davinci-002)#6Accuracy: 17.6
question-answering-on-bamboogleGoogle Search#7Accuracy: 0
question-answering-on-webquestionsSelf-Ask#20EM: 31.1