| Benchmark | Model | Rank | Results |
|---|---|---|---|
| code-generation-on-apps | LPW (GPT-4o) | #1 | Introductory Pass@1: 87.2Interview Pass@1: 65.2… |
| code-generation-on-codecontests | LPW (GPT-4o) | #2 | Test Set pass@1: 34.7 |
| code-generation-on-humaneval | Phi-2 | #2 | Pass@1: 98.2 |
| code-generation-on-livecodebench | LPW (GPT-4o) | #2 | Acc: 59.3 |
| code-generation-on-mbpp | LPW (GPT-4o) | #8 | Accuracy: 84.8 |