UniLiP: Adapting CLIP for Unified Multimodal Understanding, Generation and Editing

Benchmark Model Rank Results
text-to-image-generation-on-wiseUniLIP-3B#7Overall: 0.63Cultural: 0.66Biology: 0.60
text-to-image-generation-on-wiseUniLIP-1B#9Overall: 0.56Cultural: 0.54Biology: 0.50
image-editing-on-imgedit-dataUniLIP-3B#12Overall: 3.94Add: 4.29Adjust: 3.90Replace: 4.44Remove: 4.10…
image-editing-on-imgedit-dataUniLIP-1B#15Overall: 3.81Add: 4.11Adjust: 3.58Replace: 4.30Remove: 3.97…
text-to-image-generation-on-genevalUniLIP-3B#20Overall: 0.90
text-to-image-generation-on-genevalUniLIP-1B#38Overall: 0.88
image-generation-on-wiseUniLIP-3B#26Overall: 0.63Cultural: 0.66Biology: 0.60
image-generation-on-wiseUniLIP-1B#32Overall: 0.56Cultural: 0.54Biology: 0.50
visual-question-answering-on-mm-vetUniLIP-3B#37GPT-4 score: 62.2
visual-question-answering-on-mm-vetUniLIP-1B#48GPT-4 score: 59.4