Llama 3 Meets MoE: Efficient Upcycling
Open paper
Benchmark
Model
Rank
Results
multi-task-language-understanding-on-mmlu
Llama 3.1 (405B)
#1
Average (%): 86.6
multi-task-language-understanding-on-mmlu
Llama 3.1 (70B)
#2
Average (%): 86.0
Rank counts only results with a code link.