Mixture-of-Subspaces in Low-Rank Adaptation

Benchmark Model Rank Results
visual-question-answering-on-mmbenchLLaVA-InternLM2-ViT + MoSLoRA#1GPT-3.5 score: 73.8
visual-question-answering-on-mmbenchLLaVA-LLaMA3-8B-ViT + MoSLoRA#3GPT-3.5 score: 73.0
common-sense-reasoning-on-arc-easyLLaMA 3 8B+MoSLoRA (fine-tuned)#2Accuracy: 90.5
question-answering-on-openbookqaLLaMA-3 8B+MoSLoRA#3Accuracy: 86.8
question-answering-on-piqaLLaMA3 8B+MoSLoRA#3Accuracy: 89.7
common-sense-reasoning-on-arc-challengeLLaMA 3 8B + MoSLoRA (fine-tuned)#5Accuracy: 81.5
question-answering-on-social-iqaLLaMA-3 8B+MoSLoRA (fine-tuned)#5Accuracy: 81.0
sentence-completion-on-hellaswagLLaMA3+MoSLoRA#5Accuracy: 95.0
common-sense-reasoning-on-winograndeLLaMA3 8B+MoSLoRA#13Accuracy: 85.8
question-answering-on-boolqLLaMA3+MoSLoRA#60Accuracy: 74.6
visual-question-answering-on-mm-vetLLaVA-InternLM2-7B-ViT + MoSLoRA#139GPT-4 score: 35.2
visual-question-answering-on-mm-vetInternLM2+ViT (QMoSLoRA)#140GPT-4 score: 35.2