DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model

Benchmark Model Rank Results
question-answering-on-triviaqaDeepSeek-V2 Chat (RL) (5-shot)#1EM: 86.7
instruction-following-on-mtbenchDeepSeek-V2 Chat (RL)#4Score (1-10): 8.97
instruction-following-on-alignbenchDeepSeek-V2 Chat (RL)#5Score (0-10): 7.91
question-answering-on-natural-questionsDeepSeek-V2 Chat (RL) (5-shot)#8EM: 53.4
multi-task-language-understanding-on-mmluDeepSeek-V2 Chat (RL) (5-shot)#13Average (%): 77.8
instruction-following-on-ifevalDeepSeek-V2 Chat (RL) (0-shot)#18Prompt-level loose-accuracy: 63.8
arithmetic-reasoning-on-gsm8kDeepSeek-V2 Chat (RL) (8-shot)#51Accuracy: 92.2
code-generation-on-mbppDeepSeek-V2 Chat (RL) (3-shot)#51Accuracy: 72
code-generation-on-humanevalDeepSeek-V2 Chat (RL) (0-shot)#54Pass@1: 81.1
math-word-problem-solving-on-mathDeepSeek-V2 Chat (RL) (4-shot)#72Accuracy: 53.9