| Benchmark | Model | Rank | Results |
|---|---|---|---|
| code-generation-on-apps | CodeRL+CodeT5 | #10 | Introductory Pass@1: 20Interview Pass@1: 13.5Competition Pass@1: 33.3 |
| code-generation-on-apps | GPT-J 6B (Finetuned) | #11 | Introductory Pass@1: 6.77%Interview Pass@1: 1.80%… |
| code-generation-on-apps | GPT-Neo 2.7B (Finetuned) | #13 | Introductory Pass@1: 4.14%Interview Pass@1: 0.14%… |
| code-generation-on-apps | GPT2 1.5B (Finetuned) | #15 | Introductory Pass@1: 3.90%Interview Pass@1: 0.57%… |