Beyond Question-Based Biases: Assessing Multimodal Shortcut Learning in Visual Question Answering

Benchmark Model Rank Results
visual-question-answering-on-vqa-ceRandImg#1Accuracy (Counterexamples): 34.41
visual-question-answering-on-vqa-ceLMH + CSS#2Accuracy (Counterexamples): 34.36
visual-question-answering-on-vqa-ceLFF#3Accuracy (Counterexamples): 34.27
visual-question-answering-on-vqa-ceLMH#4Accuracy (Counterexamples): 34.26
visual-question-answering-on-vqa-ceUpDown#5Accuracy (Counterexamples): 33.91
visual-question-answering-on-vqa-ceESR#6Accuracy (Counterexamples): 33.26
visual-question-answering-on-vqa-ceLMH + RMFE#7Accuracy (Counterexamples): 33.14
visual-question-answering-on-vqa-ceBLOCK#8Accuracy (Counterexamples): 32.91
visual-question-answering-on-vqa-ceRUBi#9Accuracy (Counterexamples): 32.25