Visual Instruction Tuning

Benchmark Model Rank Results
3d-question-answering-3d-qa-on-scanqa-test-wLL3DA#16BLEU-4: 13.5ROUGE: 37.3METEOR: 15.9CIDEr: 76.8
image-classification-on-coloninst-v1-seenLLaVA-v1 (w/ LoRA, w/ extra data)#15Accuray: 89.61
image-classification-on-coloninst-v1-seenLLaVA-v1 (w/ LoRA, w/o extra data)#16Accuray: 87.86
image-classification-on-coloninst-v1-unseenLLaVA-v1 (w/ LoRA, w/o extra data)#15Accuray: 72.08
image-classification-on-coloninst-v1-unseenLLaVA-v1 (w/ LoRA, w/ extra data)#17Accuray: 42.17
mmr-total-on-mrr-benchmarkLLaVA-NEXT-34B#2Total Column Score: 412
mmr-total-on-mrr-benchmarkLLaVA-NEXT-13B#5Total Column Score: 335
mmr-total-on-mrr-benchmarkLLaVA-1.5-13B#7Total Column Score: 243
referring-expression-generation-on-coloninstLLaVA-v1 (w/ LoRA, w/ extra data)#16Accuray: 86.87
referring-expression-generation-on-coloninstLLaVA-v1 (w/ LoRA, w/o extra data)#17Accuray: 84.55
referring-expression-generation-on-coloninst-1LLaVA-v1 (w/ LoRA, w/o extra data)#16Accuray: 68.11
referring-expression-generation-on-coloninst-1LLaVA-v1 (w/ LoRA, w/ extra data)#17Accuray: 46.85
spatial-reasoning-on-embspatial-benchLLaVA-1.6#4Generation: 35.19
video-question-answering-on-mvbenchLLaVa#17Avg.: 36.0
visual-question-answering-on-benchlmmLLaVA-1.5-7B#4GPT-3.5 score: 46.83
visual-question-answering-on-benchlmmLLaVA-1-13B#7GPT-3.5 score: 43.50