| code-generation-on-apps | deepseek-ai/deepseek-coder-6.7b-instruct | #5 | Introductory Pass@1: 33.80Interview Pass@1: 19.70… |
| code-generation-on-mbpp | GPT-4 (few-shot) | #34 | Accuracy: 80 |
| code-generation-on-mbpp | GPT-3.5 Turbo (few-shot) | #55 | Accuracy: 70.8 |
| code-generation-on-mbpp | DeepSeek-Coder-Instruct 33B (few-shot) | #58 | Accuracy: 70 |
| code-generation-on-mbpp | DeepSeek-Coder-Base 33B (few-shot) | #70 | Accuracy: 66 |
| code-generation-on-mbpp | DeepSeek-Coder-Instruct 6.7B (few-shot) | #73 | Accuracy: 65.4 |
| code-generation-on-mbpp | DeepSeek-Coder-Base 6.7B (few-shot) | #87 | Accuracy: 60.6 |
| code-generation-on-mbpp | DeepSeek-Coder-Instruct 1.3B (few-shot) | #108 | Accuracy: 49.4 |
| code-generation-on-mbpp | DeepSeek-Coder-Base 1.3B (few-shot) | #122 | Accuracy: 46.2 |