| code-generation-on-mbpp | GPT-4 (Self-Debugging with unit tests + trace) | #32 | Accuracy: 80.2 |
| code-generation-on-mbpp | GPT-3.5 Turbo (Self-Debugging with unit tests + trace) | #49 | Accuracy: 72.8 |
| code-generation-on-mbpp | code-davinci-002 175B (Self-Debugging with unit tests + trace) | #54 | Accuracy: 70.8 |
| code-generation-on-mbpp | GPT-3.5 Turbo (3-shot) | #65 | Accuracy: 67.6 |
| code-generation-on-mbpp | code-davinci-002 175B (3-shot) | #83 | Accuracy: 61.4 |
| code-generation-on-mbpp | StarCoder 15.5B (Self-Debugging with unit tests + trace) | #100 | Accuracy: 53.2 |
| code-generation-on-mbpp | StarCoder 15.5B (3-shot) | #117 | Accuracy: 47.2 |