Llama 2: Open Foundation and Fine-Tuned Chat Models

Benchmark Model Rank Results
multiple-choice-question-answering-mcqa-on-agieval-english-3-5-shotLlama 2 70B (3-5 shot)#2Accuracy: 54.2
question-answering-on-triviaqaLLaMA 2 70B (one-shot)#3EM: 85
multiple-choice-question-answering-mcqa-on-mmlu-professional-medicineLlama2-7B#5Accuracy: 43.38
multiple-choice-question-answering-mcqa-on-mmlu-professional-medicineLlama2-7B-chat#6Accuracy: 40.07
social-commonsense-reasoning-on-siqaLlama 2 34B#8Accuracy: 50.9
question-answering-on-multitqLLaMA2#9Hits@1: 18.5
math-word-problem-solving-on-svampLLaMA 2-Chat#11Execution Accuracy: 69.2
math-word-problem-solving-on-mawpsLLaMA 2-Chat#15Accuracy (%): 82.4
question-answering-on-boolqLLaMA 2 70B (0-shot)#25Accuracy: 85
question-answering-on-boolqLLaMA 2 34B (0-shot)#32Accuracy: 83.7
question-answering-on-boolqLLaMA 2 13B (0-shot)#37Accuracy: 81.7
question-answering-on-boolqLLaMA 2 7B (zero-shot)#50Accuracy: 77.4
question-answering-on-piqaLLaMA 2 70B (0-shot)#30Accuracy: 82.8
question-answering-on-piqaLLaMA 2 34B (0-shot)#40Accuracy: 81.9
question-answering-on-piqaLLaMA 2 13B (0-shot)#58Accuracy: 80.5
question-answering-on-piqaLLaMA 2 7B (0-shot)#70Accuracy: 78.8
sentence-completion-on-hellaswagLLaMA 2 70B (0-shot)#37Accuracy: 85.3
sentence-completion-on-hellaswagLLaMA 2 34B (0-shot)#51Accuracy: 83.3
sentence-completion-on-hellaswagLLaMA 2 13B (0-shot)#69Accuracy: 80.7
sentence-completion-on-hellaswagLLaMA 2 7B (0-shot)#84Accuracy: 77.2
question-answering-on-natural-questionsLLaMA 2 70B (one-shot)#40EM: 33.0
multi-task-language-understanding-on-mmluLLaMA 2 34B (5-shot)#46Average (%): 62.6
multi-task-language-understanding-on-mmluLLaMA 2 13B (5-shot)#57Average (%): 54.8
multi-task-language-understanding-on-mmluLLaMA 2 7B (5-shot)#68Average (%): 45.3
code-generation-on-humanevalLlama 2 70B#123Pass@1: 29.9
code-generation-on-mbppLlama 2 70B (zero-shot)#125Accuracy: 45
code-generation-on-mbppLlama 2 34B (0-shot)#148Accuracy: 33
code-generation-on-mbppLlama 2 13B (0-shot)#149Accuracy: 30.6
code-generation-on-mbppLlama 2 7B (0-shot)#159Accuracy: 20.8
arithmetic-reasoning-on-gsm8kLLaMA 2 70B (on-shot)#222Accuracy: 56.8Parameters (Billion): 70