Contrastive Learning Rivals Masked Image Modeling in Fine-tuning via Feature Distillation

Benchmark Model Rank Results
image-classification-on-imagenetFD (CLIP ViT-L-336)#25Top 1 Accuracy: 89.0%Number of params: 307M
instance-segmentation-on-cocoFD-SwinV2-G#4mask AP: 55.4
object-detection-on-cocoFD-SwinV2-G#10box mAP: 64.2
semantic-segmentation-on-ade20kFD-SwinV2-G#9Validation mIoU: 61.4Params (M): 3000
semantic-segmentation-on-ade20k-valFD-SwinV2-G#4mIoU: 61.4