Introducing Llama 3.1: Our most capable models to date
Open paper
Benchmark
Model
Rank
Results
code-generation-on-humaneval
Llama 3.1 (405B, Instruct)
#16
Pass@1: 89.0
code-generation-on-humaneval
Llama 3.1 (70B, Instruct)
#32
Pass@1: 80.5
code-generation-on-swe-bench-pro
llama3-1-405b-instruct
#20
Resolved: 11.18
Rank counts only results with a code link.