| code-generation-on-mbpp | GPT-4 (Self-Debugging with unit tests + trace) | #15 | Accuracy: 80.2 |
| code-generation-on-mbpp | GPT-3.5 Turbo (Self-Debugging with unit tests + trace) | #17 | Accuracy: 72.8 |
| code-generation-on-mbpp | code-davinci-002 175B (Self-Debugging with unit tests + trace) | #18 | Accuracy: 70.8 |
| code-generation-on-mbpp | GPT-3.5 Turbo (3-shot) | #24 | Accuracy: 67.6 |
| code-generation-on-mbpp | code-davinci-002 175B (3-shot) | #35 | Accuracy: 61.4 |
| code-generation-on-mbpp | StarCoder 15.5B (Self-Debugging with unit tests + trace) | #44 | Accuracy: 53.2 |
| code-generation-on-mbpp | StarCoder 15.5B (3-shot) | #59 | Accuracy: 47.2 |