DeepSeek LLM: Scaling Open-Source Language Models with Longtermism

Benchmark Model Rank Results
question-answering-on-natural-questionsDeepSeek LLM 67B Chat#13EM: 47.0
common-sense-reasoning-on-arc-easyDeepSeek LLM 67B Chat#16Accuracy: 81.6
instruction-following-on-ifevalDeepSeek LLM 67B Chat#19Prompt-level loose-accuracy: 59.1
multi-task-language-understanding-on-c-evalDeepSeek LLM 67B Chat#23Accuracy: 65.2
question-answering-on-piqaDeepSeek LLM 67B Chat#29Accuracy: 82.6
common-sense-reasoning-on-winograndeDeepSeek LLM 67B Chat#53Accuracy: 76.0
code-generation-on-humanevalDeepSeek LLM 67B Chat#76Pass@1: 73.8
code-generation-on-mbppDeepSeek LLM 67B Chat#84Accuracy: 61.4
sentence-completion-on-hellaswagDeepSeek LLM 67B Chat#92Accuracy: 75.7
arithmetic-reasoning-on-gsm8kDeepSeek LLM 67B Chat#126Accuracy: 84.1
math-word-problem-solving-on-mathDeepSeek LLM 67B Chat#131Accuracy: 32.6