UniLiP: Adapting CLIP for Unified Multimodal Understanding, Generation and Editing

Benchmark Model Rank Results
image-editing-on-imgedit-dataUniLIP-3B#2Overall: 3.94Add: 4.29Adjust: 3.90Replace: 4.44Remove: 4.10…
image-editing-on-imgedit-dataUniLIP-1B#3Overall: 3.81Add: 4.11Adjust: 3.58Replace: 4.30Remove: 3.97…
image-generation-on-wiseUniLIP-3B#5Overall: 0.63Cultural: 0.66Biology: 0.60
image-generation-on-wiseUniLIP-1B#6Overall: 0.56Cultural: 0.54Biology: 0.50
text-to-image-generation-on-genevalUniLIP-3B#2Overall: 0.90
text-to-image-generation-on-genevalUniLIP-1B#3Overall: 0.88
text-to-image-generation-on-wiseUniLIP-3B#3Overall: 0.63Cultural: 0.66Biology: 0.60
text-to-image-generation-on-wiseUniLIP-1B#4Overall: 0.56Cultural: 0.54Biology: 0.50
visual-question-answering-on-mm-vetUniLIP-3B#25GPT-4 score: 62.2
visual-question-answering-on-mm-vetUniLIP-1B#33GPT-4 score: 59.4