Robust Adaptation of Large Multimodal Models for Retrieval Augmented Hateful Meme Detection

Benchmark Model Rank Results
hateful-meme-classification-on-harm-pRA-HMD (Qwen2-VL-7B)#1Accuracy: 91.6F1: 91.1
hateful-meme-classification-on-harmemeRA-HMD (Qwen2VL-7B)#1AUROC: 93.2Accuracy: 88.1
hateful-meme-classification-on-harmemeRA-HMD (Qwen2VL-2B)#2AUROC: 92.9Accuracy: 87.7
hateful-meme-classification-on-pridemmRA-HMD (Qwen2-VL-7B)#1Accuracy: 78.1F1: 78.4
hateful-meme-classification-on-pridemmRA-HMD (Qwen2-VL-2B)#5Accuracy: 76.0F1: 76.7
meme-classification-on-hateful-memesRA-HMD (Qwen2-VL-7B)#1ROC-AUC: 0.911Accuracy: 0.821
meme-classification-on-hateful-memesRA-HMD (LLaVA-1.5-7B)#2ROC-AUC: 0.897Accuracy: 0.809
meme-classification-on-hateful-memesRA-HMD (Qwen2-VL-2B)#3ROC-AUC: 0.884Accuracy: 0.791
meme-classification-on-multioffRA-HMD (Qwen2-VL-7B)#1Accuracy: 71.1F1: 64.8