Nemotron-4 15B Technical Report

Benchmark Model Rank Results
social-commonsense-reasoning-on-siqaNemotron-4 15B#1Accuracy: 60.9
cross-lingual-question-answering-on-tydiqa-goldpNemotron-4 15B (1-shot)#6EM: 50.5
common-sense-reasoning-on-arc-easyNemotron-4 15B#18Accuracy: 80.9
question-answering-on-piqaNemotron-4 15B#30Accuracy: 82.4
common-sense-reasoning-on-winograndeNemotron-4 15B#40Accuracy: 78.0
multi-task-language-understanding-on-mmluNemotron-4 15B#42Average (%): 64.2
sentence-completion-on-hellaswagNemotron-4 15B#58Accuracy: 82.4
code-generation-on-humanevalNemotron-4 15B#116Pass@1: 31.6
code-generation-on-mbppNemotron-4 15B (3-shot)#135Accuracy: 40.6
arithmetic-reasoning-on-gsm8kNemotron-4 15B (8-shot)#235Accuracy: 46.0