Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time

Open paper
Benchmark Model Rank Results
domain-generalization-on-imagenet-aModel soups (BASIC-L)#1Top-1 accuracy %: 94.17
domain-generalization-on-imagenet-aModel soups (ViT-G/14)#2Top-1 accuracy %: 92.67
domain-generalization-on-imagenet-rModel soups (BASIC-L)#1Top-1 Error Rate: 3.90
domain-generalization-on-imagenet-rModel soups (ViT-G/14)#2Top-1 Error Rate: 4.54
domain-generalization-on-imagenet-sketchModel soups (BASIC-L)#1Top-1 accuracy: 77.18
domain-generalization-on-imagenet-sketchModel soups (ViT-G/14)#2Top-1 accuracy: 74.24
image-classification-on-imagenetModel soups (BASIC-L)#2Top 1 Accuracy: 90.98%Number of params: 2440M
image-classification-on-imagenetModel soups (ViT-G/14)#3Top 1 Accuracy: 90.94%Number of params: 1843M
image-classification-on-imagenet-realBaseline (ViT-G/14)#1Accuracy: 91.78%
image-classification-on-imagenet-realModel soups (ViT-G/14)#3Accuracy: 91.20%Params: 1843M
image-classification-on-imagenet-realModel soups (BASIC-L)#7Accuracy: 91.03%Params: 2440M
image-classification-on-imagenet-v2Model soups (BASIC-L)#1Top 1 Accuracy: 84.63
image-classification-on-imagenet-v2Model soups (ViT-G/14)#3Top 1 Accuracy: 84.22
image-classification-on-objectnetBaseline (ViT-G/14)#4Top-1 Accuracy: 79.03
image-classification-on-objectnetModel soups (ViT-G/14)#5Top-1 Accuracy: 78.52
unsupervised-domain-adaptation-on-imagenet-rModel soups (ViT-G/14)#1Top 1 Error: 4.54

Rank counts only results with a code link.