| code-generation-on-apps | deepseek-ai/deepseek-coder-6.7b-instruct | #5 | Introductory Pass@1: 33.80Interview Pass@1: 19.70… |
| code-generation-on-mbpp | GPT-4 (few-shot) | #16 | Accuracy: 80 |
| code-generation-on-mbpp | GPT-3.5 Turbo (few-shot) | #19 | Accuracy: 70.8 |
| code-generation-on-mbpp | DeepSeek-Coder-Instruct 33B (few-shot) | #20 | Accuracy: 70 |
| code-generation-on-mbpp | DeepSeek-Coder-Base 33B (few-shot) | #28 | Accuracy: 66 |
| code-generation-on-mbpp | DeepSeek-Coder-Instruct 6.7B (few-shot) | #30 | Accuracy: 65.4 |
| code-generation-on-mbpp | DeepSeek-Coder-Base 6.7B (few-shot) | #38 | Accuracy: 60.6 |
| code-generation-on-mbpp | DeepSeek-Coder-Instruct 1.3B (few-shot) | #51 | Accuracy: 49.4 |
| code-generation-on-mbpp | DeepSeek-Coder-Base 1.3B (few-shot) | #63 | Accuracy: 46.2 |