| Benchmark | Model | Rank | Results |
|---|---|---|---|
| code-generation-on-mbpp | Bard (PaLM 2/chat-bison-001) | – | Accuracy: 76.2 |
| code-generation-on-mbpp | Claude | – | Accuracy: 71.4 |
| code-generation-on-mbpp | GPT-3.5 Turbo (ChatGPT) | – | Accuracy: 83.2 |
| code-generation-on-mbpp | GPT-4 (Bing Chat) | – | Accuracy: 82 |
| code-generation-on-mbpp | GPT-4 (ChatGPT Plus) | – | Accuracy: 87.5 |