Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs

Benchmark Model Rank Results
science-question-answering-on-scienceqaPhi-4-Multimodal#2Avg. Accuracy: 97.5
math-word-problem-solving-on-mgsm-0-shotPhi-4-Mini (0-shot, CoT)#5Accuracy: 63.9
visual-question-answering-vqa-on-infographicvqaPhi-4-Multimodal#12ANLS: 72.7
multi-task-language-understanding-on-mmmluPhi-4-Mini (5-shot)#15Accuracy: 49.3
visual-question-answering-on-docvqa-testPhi-4-Multimodal#22ANLS: 0.932
object-hallucination-evaluation-on-popePhi-4-Multimodal#27F1: 85.6
multi-task-language-understanding-on-mmluPhi-4-Mini (5-shot)#35Average (%): 67.3
optical-character-recognition-ocr-on-ocrbenchPhi-4-Multimodal#40Score: 844
question-answering-on-boolqPhi-4-Mini (2-shot)#41Accuracy: 81.2
math-word-problem-solving-on-mathPhi-4-Mini (0-shot, CoT)#44Accuracy: 64.0
multimodal-reasoning-on-mmmu-proPhi-4-Multimodal#53Accuracy: 38.5
mathematical-reasoning-on-math-500Phi-4-Mini reasoning#57Accuracy: 90.4
chart-question-answering-on-chartqaPhi-4-Multimodal#651:1 Accuracy: 81.4
code-generation-on-mbppPhi-4-Mini (3-shot)#75Accuracy: 65.3
multimodal-reasoning-on-mmmuPhi-4-Multimodal#75Accuracy: 55.1
code-generation-on-humanevalPhi-4-Mini#76Pass@1: 74.4
question-answering-on-piqaPhi-4-Mini (5-shot)#82Accuracy: 77.6
mathematical-reasoning-on-aime24Phi-4-Mini reasoning#84Acc: 50.0
arithmetic-reasoning-on-gsm8kPhi-4-Mini (8-shot, CoT)#89Accuracy: 88.6
common-sense-reasoning-on-winograndePhi-4-Mini (5-shot)#89Accuracy: 67.0
multimodal-reasoning-on-mathvistaPhi-4-Multimodal#91Accuracy: 62.4
multi-task-language-understanding-on-mmlu-proPhi-4-Mini (0-shot, CoT)#112Accuracy: 52.8
sentence-completion-on-hellaswagPhi-4-Mini (5-shot)#113Accuracy: 69.1
multiple-choice-question-answering-mcqa-on-gpqa-diamondPhi-4-Mini reasoning#118Accuracy: 49.0
speech-recognition-on-amimicrosoft/Phi-4-multimodal-instruct–Word Error Rate (WER): 10.69
speech-recognition-on-ami-cleanedmicrosoft/Phi-4-multimodal-instruct–Word Error Rate (WER): 9.11
speech-recognition-on-earnings-22microsoft/Phi-4-multimodal-instruct–Word Error Rate (WER): 9.35
speech-recognition-on-gigaspeech-cleanedmicrosoft/Phi-4-multimodal-instruct–Word Error Rate (WER): 7.8
speech-recognition-on-gigaspeech-testmicrosoft/Phi-4-multimodal-instruct–Word Error Rate (WER): 7.91
speech-recognition-on-librispeech-test-cleanmicrosoft/Phi-4-multimodal-instruct–Word Error Rate (WER): 1.35
speech-recognition-on-librispeech-test-othermicrosoft/Phi-4-multimodal-instruct–Word Error Rate (WER): 3.45
speech-recognition-on-spgispeechmicrosoft/Phi-4-multimodal-instruct–Word Error Rate (WER): 2.73
speech-recognition-on-voxpopulimicrosoft/Phi-4-multimodal-instruct–Word Error Rate (WER): 5.77
speech-recognition-on-voxpopuli-cleanedmicrosoft/Phi-4-multimodal-instruct–Word Error Rate (WER): 4.18