Coarse-to-Fine Contrastive Learning in Image-Text-Graph Space for Improved Vision-Language Compositionality

Open paper
Benchmark Model Rank Results
image-retrieval-on-crepe-vision-languageCLIP (CC-FT)–Recall@1 (HN-Atom, UC): 35.0Recall@1 (HN-Comp, UC): 45.1
image-retrieval-on-crepe-vision-languageCLIP (YFCC-FT)–Recall@1 (HN-Atom, UC): 39.5Recall@1 (HN-Comp, UC): 39.8
image-retrieval-on-crepe-vision-languageCLIP-FT (CC-FT)–Recall@1 (HN-Atom, UC): 35.6Recall@1 (HN-Comp, UC): 45.8
image-retrieval-on-crepe-vision-languageCLIP-FT (YFCC-FT)–Recall@1 (HN-Atom, UC): 38.3Recall@1 (HN-Comp, UC): 36.4
image-retrieval-on-crepe-vision-languageMosaiCLIP (CC-FT)–Recall@1 (HN-Atom, UC): 40.9Recall@1 (HN-Comp, UC): 72.4
image-retrieval-on-crepe-vision-languageMosaiCLIP (YFCC-FT)–Recall@1 (HN-Atom, UC): 41.5Recall@1 (HN-Comp, UC): 48.8
image-retrieval-on-crepe-vision-languageNegCLIP (CC-FT)–Recall@1 (HN-Atom, UC): 37.5Recall@1 (HN-Comp, UC): 53.1
image-retrieval-on-crepe-vision-languageNegCLIP (YFCC-FT)–Recall@1 (HN-Atom, UC): 39.0Recall@1 (HN-Comp, UC): 38.8
image-retrieval-on-crepe-vision-languageRN-50 (CLIP, CC-12M)–Recall@1 (HN-Atom, UC): 36.7Recall@1 (HN-Comp, UC): 42.9
image-retrieval-on-crepe-vision-languageRN-50 (MosaiCLIP, CC-12M)–Recall@1 (HN-Atom, UC): 44.4Recall@1 (HN-Comp, UC): 92.6
image-retrieval-on-crepe-vision-languageRN-50 (NegCLIP, CC-12M)–Recall@1 (HN-Atom, UC): 41.4Recall@1 (HN-Comp, UC): 82.0
image-retrieval-on-crepe-vision-languageSwin-T (CLIP, CC-12M)–Recall@1 (HN-Atom, UC): 37.3Recall@1 (HN-Comp, UC): 44.1
image-retrieval-on-crepe-vision-languageSwin-T (MosaiCLIP, CC-12M)–Recall@1 (HN-Atom, UC): 44.5Recall@1 (HN-Comp, UC): 92.1
image-retrieval-on-crepe-vision-languageSwin-T (NegCLIP, CC-12M)–Recall@1 (HN-Atom, UC): 39.6Recall@1 (HN-Comp, UC): 80.3

Rank counts only results with a code link.