| Benchmark | Model | Rank | Results |
|---|---|---|---|
| code-generation-on-pecc | Claude 3 Haiku | #1 | Pass@3: 27.67 |
| code-generation-on-pecc | GPT-3.5 Turbo | #2 | Pass@3: 23.75 |
| code-generation-on-pecc | codechat-bison | #3 | Pass@3: 11.39 |
| code-generation-on-pecc | chat-bison | #4 | Pass@3: 8.48 |
| code-generation-on-pecc | Mixtral-8x7B-Instruct | #5 | Pass@3: 8.35 |
| code-generation-on-pecc | Phi-3-mini-128k-instruct | #6 | Pass@3: 7.18 |
| code-generation-on-pecc | WizardLM-2-7B | #7 | Pass@3: 3.72 |
| code-generation-on-pecc | Llama-3-8B-Instruct | #8 | Pass@3: 3.1 |