Improved Large Language Diffusion Models

Benchmark Model Rank Results
multiple-choice-question-answering-mcqa-on-arc-ciLLaDA 8B Base#3Accuracy: 60.8
multi-task-language-understanding-on-mmluiLLaDA 8B Base#19Average (%): 74.8
multi-task-language-understanding-on-mmluiLLaDA 8B Instruct#25Average (%): 71.6
multi-task-language-understanding-on-mmlu-reduxiLLaDA 8B Instruct#31Accuracy: 76.4
math-word-problem-solving-on-mathiLLaDA 8B Instruct#64Accuracy: 56.7
math-word-problem-solving-on-mathiLLaDA 8B Base#122Accuracy: 38.4
question-answering-on-piqaConfidence-based scoring#67Accuracy: 78.5
question-answering-on-piqaiLLaDA 8B Base#68Accuracy: 78.5
arithmetic-reasoning-on-gsm8kiLLaDA 8B Instruct#82Accuracy: 89.0
arithmetic-reasoning-on-gsm8kiLLaDA 8B Base#142Accuracy: 81.9
sentence-completion-on-hellaswagiLLaDA 8B Base#85Accuracy: 76.6
code-generation-on-humanevaliLLaDA 8B Instruct#91Pass@1: 65.9
code-generation-on-humanevaliLLaDA 8B Base#101Pass@1: 50
code-generation-on-mbppiLLaDA 8B Instruct#91Accuracy: 58
code-generation-on-mbppiLLaDA 8B Base#92Accuracy: 57.8
multi-task-language-understanding-on-mmlu-proiLLaDA 8B Instruct#111Accuracy: 52.3