Youtu-LLM: Unlocking the Native Agentic Potential for Lightweight Large Language Models

Benchmark Model Rank Results
question-answering-on-drop-3-shotYoutu-LLM 2B (3-shot)#3F1: 86.7
agents-on-gaia-text-only-103-validation-tasksYoutu-LLM 2B#8Accuracy: 33.9
instruction-following-on-ifevalYoutu-LLM 2B#10Prompt-level strict-accuracy: 81.2
code-generation-on-humanevalYoutu-LLM 2B#12Pass@1: 95.9
code-generation-on-mbppYoutu-LLM 2B#17Accuracy: 85
long-context-understanding-on-longbench-v2Youtu-LLM (2B, base)#19Accuracy: 27.2
agents-on-benchYoutu-LLM 2B#30Accuracy: 15.0
multi-task-language-understanding-on-mmlu-reduxYoutu-LLM 2B#32Accuracy: 75.8
mathematical-reasoning-on-math-500Youtu-LLM 2B#40Accuracy: 93.7
code-generation-on-livecodebench-v6Youtu-LLM 2B#55Pass@1: 43.7
mathematical-reasoning-on-aime24Youtu-LLM 2B#59Acc: 65.4
mathematical-reasoning-on-aime-2025Youtu-LLM 2B#86Accuracy: 49.8
code-generation-on-swe-bench-verifiedYoutu-LLM 2B#87Accuracy: 17.7
multi-task-language-understanding-on-mmlu-proYoutu-LLM 2B (5-shot)#92Accuracy: 61.6
math-word-problem-solving-on-mathYoutu-LLM (2B, base)#108Accuracy: 44.4
multiple-choice-question-answering-mcqa-on-gpqa-diamondYoutu-LLM 2B#117Accuracy: 48.0
arithmetic-reasoning-on-gsm8kYoutu-LLM (2B, base)#169Accuracy: 77.6