Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time

Open paper
Benchmark Model Rank Results
domain-generalization-on-imagenet-aModel soups (BASIC-L)#1Top-1 accuracy %: 94.17
domain-generalization-on-imagenet-aModel soups (ViT-G/14)#2Top-1 accuracy %: 92.67
domain-generalization-on-imagenet-rModel soups (BASIC-L)#1Top-1 Error Rate: 3.90
domain-generalization-on-imagenet-rModel soups (ViT-G/14)#2Top-1 Error Rate: 4.54
domain-generalization-on-imagenet-sketchModel soups (BASIC-L)#1Top-1 accuracy: 77.18
domain-generalization-on-imagenet-sketchModel soups (ViT-G/14)#2Top-1 accuracy: 74.24
image-classification-on-imagenet-realBaseline (ViT-G/14)#1Accuracy: 91.78%
image-classification-on-imagenet-realModel soups (ViT-G/14)#3Accuracy: 91.20%Params: 1843M
image-classification-on-imagenet-realModel soups (BASIC-L)#8Accuracy: 91.03%Params: 2440M
image-classification-on-imagenet-v2Model soups (BASIC-L)#1Top 1 Accuracy: 84.63
image-classification-on-imagenet-v2Model soups (ViT-G/14)#3Top 1 Accuracy: 84.22
unsupervised-domain-adaptation-on-imagenet-rModel soups (ViT-G/14)#1Top 1 Error: 4.54
image-classification-on-imagenetModel soups (BASIC-L)#2Top 1 Accuracy: 90.98%Number of params: 2440M
image-classification-on-imagenetModel soups (ViT-G/14)#3Top 1 Accuracy: 90.94%Number of params: 1843M
natural-language-inference-on-rte-glue-devT5-large (Greedy soup)#7Accuracy: 84.8
image-classification-on-objectnetBaseline (ViT-G/14)#8Top-1 Accuracy: 79.03
image-classification-on-objectnetModel soups (ViT-G/14)#10Top-1 Accuracy: 78.52
linguistic-acceptability-on-cola-glue-devBERT-large (Greedy soup)#8Matthews Correlation: 63.1
sentiment-analysis-on-sst-2-binary-classificationT5-large (Greedy soup)#88Dev Accuracy: 96.3

Rank counts only results with a code link.