Composing Ensembles of Pre-trained Models via Iterative Consensus

Benchmark Model Rank Results
arithmetic-reasoning-on-gsm8kGPT-2-Medium 355M (BS=5)Accuracy: 12.2Parameters (Billion): 0.355
arithmetic-reasoning-on-gsm8kGPT-2-Medium 355M (fine-tuned, BS=5)Accuracy: 18.3Parameters (Billion): 0.355
arithmetic-reasoning-on-gsm8kGPT-2-Medium 355M + question-solution classifier (BS=1)Accuracy: 16.8Parameters (Billion): 0.355
arithmetic-reasoning-on-gsm8kGPT-2-Medium 355M + question-solution classifier (BS=5)Accuracy: 20.8Parameters (Billion): 0.355
image-generation-on-imagenet-64x64GLIDE + CLIPFID: 30.462Inception Score: 25.017KID: 6.174
image-generation-on-imagenet-64x64GLIDE + CLIP + CLS + CLS-FREEFID: 29.184Inception Score: 34.952KID: 3.766
image-generation-on-imagenet-64x64GLIDE + CLSFID: 30.871Inception Score: 22.077
image-generation-on-imagenet-64x64GLIDE + CLS-FREEFID: 29.219Inception Score: 25.926KID: 5.325
image-generation-on-imagenet-64x64GLIDE +CLSKID: 7.952
video-question-answering-on-activitynet-qaGPT-2 + CLIP-14 + CLIP-multilingual (Zero-Shot)Accuracy: 61.2
video-question-answering-on-activitynet-qaGPT-2 + CLIP-32 (Zero-Shot)Accuracy: 58.4