Composing Ensembles of Pre-trained Models via Iterative Consensus

Benchmark Model Rank Results
arithmetic-reasoning-on-gsm8kGPT-2-Medium 355M (BS=5)–Accuracy: 12.2Parameters (Billion): 0.355
arithmetic-reasoning-on-gsm8kGPT-2-Medium 355M (fine-tuned, BS=5)–Accuracy: 18.3Parameters (Billion): 0.355
arithmetic-reasoning-on-gsm8kGPT-2-Medium 355M + question-solution classifier (BS=1)–Accuracy: 16.8Parameters (Billion): 0.355
arithmetic-reasoning-on-gsm8kGPT-2-Medium 355M + question-solution classifier (BS=5)–Accuracy: 20.8Parameters (Billion): 0.355
image-generation-on-imagenet-64x64GLIDE + CLIP–FID: 30.462Inception Score: 25.017KID: 6.174
image-generation-on-imagenet-64x64GLIDE + CLIP + CLS + CLS-FREE–FID: 29.184Inception Score: 34.952KID: 3.766
image-generation-on-imagenet-64x64GLIDE + CLS–FID: 30.871Inception Score: 22.077
image-generation-on-imagenet-64x64GLIDE + CLS-FREE–FID: 29.219Inception Score: 25.926KID: 5.325
image-generation-on-imagenet-64x64GLIDE +CLS–KID: 7.952
video-question-answering-on-activitynet-qaGPT-2 + CLIP-14 + CLIP-multilingual (Zero-Shot)–Accuracy: 61.2
video-question-answering-on-activitynet-qaGPT-2 + CLIP-32 (Zero-Shot)–Accuracy: 58.4