CuMo: Scaling Multimodal LLM with Co-Upcycled Mixture-of-Experts

Benchmark Model Rank Results
visual-instruction-following-on-llava-benchCuMo-7B#1avg score: 85.7
visual-question-answering-on-gqa-test-devCuMo-7B#2Accuracy: 64.9
visual-question-answering-on-mm-vetCuMo-7B#45GPT-4 score: 51.0Params: 7B
visual-question-answering-on-mmbenchCuMo-7B#2GPT-3.5 score: 73.0
visual-question-answering-on-vqa-v2-test-devCuMo-7B#6Accuracy: 82.2